[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:44 dist-test-slave-1vmg krb5kdc[24907](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:44 dist-test-slave-1vmg krb5kdc[24907](info): set up 2 sockets
Aug 08 02:17:44 dist-test-slave-1vmg krb5kdc[24907](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 02:17:45.992128 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.280s user 0.005s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:17:46 dist-test-slave-1vmg krb5kdc[24907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:17:46.140524 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:46251
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:43297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:46251
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:46.274760 24923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:46.275158 24923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:46.275205 24923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:46.279574 24923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:46.279664 24923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:46.279685 24923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:46.279700 24923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:46.279716 24923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:46.285923 24923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:43297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:46251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:46251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:46.287762 24923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:46.289291 24923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:17:46.304087 24923 server_base.cc:1034] running on GCE node
W20260808 02:17:46.304112 24931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:46.304800 24930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:46.305053 24934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:46.306008 24923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:46.308269 24923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:46.309619 24923 hybrid_clock.cc:648] HybridClock initialized: now 1786155466309573 us; error 92 us; skew 500 ppm
Aug 08 02:17:46 dist-test-slave-1vmg krb5kdc[24907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155466, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:46.313344 24923 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:17:46.314899 24923 webserver.cc:460] Webserver started at http://127.24.79.254:41163/ using document root <none> and password file <none>
I20260808 02:17:46.316020 24923 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:46.316110 24923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:46.316592 24923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:46.319625 24923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "62616c78826a413ea836e4df96ebd3df"
format_stamp: "Formatted at 2026-08-08 02:17:46 on dist-test-slave-1vmg"
server_key: "83667e747de4d119aace1eda10b12e79"
server_key_iv: "f0193eef1344966824a717c64279b65c"
server_key_version: "encryptionkey@0"
I20260808 02:17:46.320317 24923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "62616c78826a413ea836e4df96ebd3df"
format_stamp: "Formatted at 2026-08-08 02:17:46 on dist-test-slave-1vmg"
server_key: "83667e747de4d119aace1eda10b12e79"
server_key_iv: "f0193eef1344966824a717c64279b65c"
server_key_version: "encryptionkey@0"
I20260808 02:17:46.326253 24923 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260808 02:17:46.330528 24941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:46.332831 24923 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260808 02:17:46.333153 24923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "62616c78826a413ea836e4df96ebd3df"
format_stamp: "Formatted at 2026-08-08 02:17:46 on dist-test-slave-1vmg"
server_key: "83667e747de4d119aace1eda10b12e79"
server_key_iv: "f0193eef1344966824a717c64279b65c"
server_key_version: "encryptionkey@0"
I20260808 02:17:46.333317 24923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:46.351518 24923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:46.356117 24923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:46.356397 24923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:46.367911 24923 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:46251
I20260808 02:17:46.367978 25027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:46251 every 8 connection(s)
I20260808 02:17:46.369544 24923 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:46.370529 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 24923
I20260808 02:17:46.370699 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:17:46.372341 24895 external_mini_cluster.cc:1428] Setting key a94c545e57cefb3380e434f03a9b0453
I20260808 02:17:46.375731 25028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:46.386579 25028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62616c78826a413ea836e4df96ebd3df: Bootstrap starting.
I20260808 02:17:46.391000 25028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62616c78826a413ea836e4df96ebd3df: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:46.392325 25028 log.cc:824] T 00000000000000000000000000000000 P 62616c78826a413ea836e4df96ebd3df: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:17:46 dist-test-slave-1vmg krb5kdc[24907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:17:46.395886 25028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62616c78826a413ea836e4df96ebd3df: No bootstrap required, opened a new log
I20260808 02:17:46.401719 25030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:46.378036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60618 (local address 127.24.79.254:46251)
0808 02:17:46.378629 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:46.378649 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:46.379931 (+ 1282us) server_negotiation.cc:407] Connection header received
0808 02:17:46.383522 (+ 3591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:46.383542 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:46.384007 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:46.384351 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:46.387093 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:46.388193 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:46.389495 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:46.389992 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:46.390169 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:46.394694 (+ 4525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:46.394739 (+ 45us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:46.394758 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:46.394799 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:46.397077 (+ 2278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:46.397727 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:46.397734 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:46.397742 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:46.397910 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:46.398349 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:46.398353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:46.398354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:46.399187 (+ 833us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:46.399422 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:46.400163 (+ 741us) server_negotiation.cc:299] Negotiation successful
0808 02:17:46.400497 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":175,"threads_started":1}
I20260808 02:17:46.402535 25028 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62616c78826a413ea836e4df96ebd3df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62616c78826a413ea836e4df96ebd3df" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46251 } }
I20260808 02:17:46.402961 25028 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62616c78826a413ea836e4df96ebd3df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:46.403075 25028 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62616c78826a413ea836e4df96ebd3df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62616c78826a413ea836e4df96ebd3df, State: Initialized, Role: FOLLOWER
I20260808 02:17:46.403973 25028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62616c78826a413ea836e4df96ebd3df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62616c78826a413ea836e4df96ebd3df" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46251 } }
I20260808 02:17:46.404213 25028 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62616c78826a413ea836e4df96ebd3df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:46.404363 25028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62616c78826a413ea836e4df96ebd3df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:46.404507 25028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62616c78826a413ea836e4df96ebd3df [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:46.405921 25028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62616c78826a413ea836e4df96ebd3df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62616c78826a413ea836e4df96ebd3df" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46251 } }
I20260808 02:17:46.406661 25028 leader_election.cc:302] T 00000000000000000000000000000000 P 62616c78826a413ea836e4df96ebd3df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62616c78826a413ea836e4df96ebd3df; no voters:
I20260808 02:17:46.407151 25028 leader_election.cc:288] T 00000000000000000000000000000000 P 62616c78826a413ea836e4df96ebd3df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:46.407352 25033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62616c78826a413ea836e4df96ebd3df [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:46.407584 25033 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62616c78826a413ea836e4df96ebd3df [term 1 LEADER]: Becoming Leader. State: Replica: 62616c78826a413ea836e4df96ebd3df, State: Running, Role: LEADER
I20260808 02:17:46.408057 25033 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62616c78826a413ea836e4df96ebd3df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62616c78826a413ea836e4df96ebd3df" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46251 } }
I20260808 02:17:46.408923 25028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62616c78826a413ea836e4df96ebd3df [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:46.410861 25037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62616c78826a413ea836e4df96ebd3df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62616c78826a413ea836e4df96ebd3df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62616c78826a413ea836e4df96ebd3df" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46251 } } }
I20260808 02:17:46.411057 25037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62616c78826a413ea836e4df96ebd3df [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:46.411736 25038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62616c78826a413ea836e4df96ebd3df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62616c78826a413ea836e4df96ebd3df. Latest consensus state: current_term: 1 leader_uuid: "62616c78826a413ea836e4df96ebd3df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62616c78826a413ea836e4df96ebd3df" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46251 } } }
I20260808 02:17:46.411847 25038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62616c78826a413ea836e4df96ebd3df [sys.catalog]: This master's current role is: LEADER
W20260808 02:17:46.417011 25055 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 62616c78826a413ea836e4df96ebd3df: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:17:46.417238 25055 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:17:46.417320 25051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:46.418277 25051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:46.427265 25051 catalog_manager.cc:1347] Generated new cluster ID: 61d6115d1f6543a09288f2ba8a2c9ed7
I20260808 02:17:46.427369 25051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:46.436389 25051 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:46.438016 25051 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:46.445420 25051 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62616c78826a413ea836e4df96ebd3df: Generated new TSK 0
I20260808 02:17:46.446547 25051 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.79.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
I20260808 02:17:46.492376 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:46251
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:43297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:46.628562 25066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:46.628887 25066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:46.628937 25066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:46.633216 25066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:46.633297 25066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:46.633361 25066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:17:46.639209 25066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:43297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:46251
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:46.640551 25066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:46.641690 25066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:46.652748 25083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:46.652827 25066 server_base.cc:1034] running on GCE node
W20260808 02:17:46.652776 25081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:46.652760 25080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:46.653596 25066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:46.654314 25066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:46.655493 25066 hybrid_clock.cc:648] HybridClock initialized: now 1786155466655464 us; error 39 us; skew 500 ppm
Aug 08 02:17:46 dist-test-slave-1vmg krb5kdc[24907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155466, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:46.658689 25066 init.cc:377] Logged in from keytab as kudu/127.24.79.193@KRBTEST.COM (short username kudu)
I20260808 02:17:46.659726 25066 webserver.cc:460] Webserver started at http://127.24.79.193:36549/ using document root <none> and password file <none>
I20260808 02:17:46.660321 25066 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:46.660384 25066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:46.660642 25066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:46.662969 25066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "c289772760f0412cb0393f0fbb6c6fb6"
format_stamp: "Formatted at 2026-08-08 02:17:46 on dist-test-slave-1vmg"
server_key: "2e7c9bb50660f5e14a1873b1a704fcad"
server_key_iv: "71636e71f1a8686da5b90ac55aeac6ba"
server_key_version: "encryptionkey@0"
I20260808 02:17:46.663585 25066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "c289772760f0412cb0393f0fbb6c6fb6"
format_stamp: "Formatted at 2026-08-08 02:17:46 on dist-test-slave-1vmg"
server_key: "2e7c9bb50660f5e14a1873b1a704fcad"
server_key_iv: "71636e71f1a8686da5b90ac55aeac6ba"
server_key_version: "encryptionkey@0"
I20260808 02:17:46.667730 25066 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:17:46.670485 25091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:46.671985 25066 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:17:46.672113 25066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "c289772760f0412cb0393f0fbb6c6fb6"
format_stamp: "Formatted at 2026-08-08 02:17:46 on dist-test-slave-1vmg"
server_key: "2e7c9bb50660f5e14a1873b1a704fcad"
server_key_iv: "71636e71f1a8686da5b90ac55aeac6ba"
server_key_version: "encryptionkey@0"
I20260808 02:17:46.672225 25066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:46.696547 25066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:46.700162 25066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:46.700346 25066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:46.701104 25066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:17:46.702454 25066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:17:46.702526 25066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:46.702590 25066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:17:46.702617 25066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:46.713145 25066 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:32957
I20260808 02:17:46.713169 25284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:32957 every 8 connection(s)
I20260808 02:17:46.714186 25066 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:17:46.724002 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 25066
I20260808 02:17:46.724110 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:17:46.724453 24895 external_mini_cluster.cc:1428] Setting key 0456b19f2c4adfcb6032599b8d2ed687
Aug 08 02:17:46 dist-test-slave-1vmg krb5kdc[24907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155466, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:17:46.731340 25030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:46.716983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:46895 (local address 127.24.79.254:46251)
0808 02:17:46.717143 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:46.717148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:46.718223 (+ 1075us) server_negotiation.cc:407] Connection header received
0808 02:17:46.720169 (+ 1946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:46.720173 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:46.720232 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:46.720322 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:46.721782 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:46.722400 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:46.723713 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:46.723973 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:46.724122 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:46.727475 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:46.727478 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:46.727480 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:46.727507 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:46.728998 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:46.729603 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:46.729606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:46.729608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:46.729698 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:46.730063 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:46.730066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:46.730067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:46.730279 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:46.730371 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:46.731004 (+ 633us) server_negotiation.cc:299] Negotiation successful
0808 02:17:46.731165 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 02:17:46.732221 25289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:46.717470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:46251 (local address 127.24.79.193:46895)
0808 02:17:46.718019 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:46.718075 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:46.719819 (+ 1744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:46.720585 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:46.720594 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:46.721025 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:46.721590 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:46.721606 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:46.722528 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:46.722530 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:46.723563 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:46.723575 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:46.724268 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:46.724274 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:46.724511 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:46.725386 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:46.725410 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:46.727270 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:46.729134 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:46.729142 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:46.729164 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:46.729483 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:46.729789 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:46.729792 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:46.729794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:46.729958 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:46.730391 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:46.730399 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:46.730701 (+ 302us) client_negotiation.cc:769] Sending connection context
0808 02:17:46.730884 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 02:17:46.731301 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":135,"threads_started":1}
I20260808 02:17:46.733565 25285 heartbeater.cc:347] Connected to a master server at 127.24.79.254:46251
I20260808 02:17:46.733875 25285 heartbeater.cc:464] Registering TS with master...
I20260808 02:17:46.734571 25285 heartbeater.cc:511] Master 127.24.79.254:46251 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.24.79.194@KRBTEST.COM; defaulting to no policy
I20260808 02:17:46.737252 24975 ts_manager.cc:194] Registered new tserver with Master: c289772760f0412cb0393f0fbb6c6fb6 (127.24.79.193:32957)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.194@KRBTEST.COM" created.
I20260808 02:17:46.739045 24975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.193@KRBTEST.COM'} at 127.24.79.193:46895
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
I20260808 02:17:46.758184 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:46251
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:43297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:46.880367 25292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:46.880671 25292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:46.880718 25292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:46.884843 25292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:46.884923 25292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:46.884984 25292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:17:46.891600 25292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:43297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:46251
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:46.892793 25292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:46.893889 25292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:46.904866 25297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:46.905033 25292 server_base.cc:1034] running on GCE node
W20260808 02:17:46.904865 25303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:46.904888 25298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:46.905709 25292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:46.906416 25292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:46.907614 25292 hybrid_clock.cc:648] HybridClock initialized: now 1786155466907587 us; error 39 us; skew 500 ppm
Aug 08 02:17:46 dist-test-slave-1vmg krb5kdc[24907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155466, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:46.910833 25292 init.cc:377] Logged in from keytab as kudu/127.24.79.194@KRBTEST.COM (short username kudu)
I20260808 02:17:46.911926 25292 webserver.cc:460] Webserver started at http://127.24.79.194:44715/ using document root <none> and password file <none>
I20260808 02:17:46.912532 25292 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:46.912604 25292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:46.912825 25292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:46.915119 25292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "a2413510665e459eba73c18ef3e30e9b"
format_stamp: "Formatted at 2026-08-08 02:17:46 on dist-test-slave-1vmg"
server_key: "78ef11e09fe071d98a8d4e80699d4a6c"
server_key_iv: "8eba0da92dbdcdf4e947e0c401636792"
server_key_version: "encryptionkey@0"
I20260808 02:17:46.915783 25292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "a2413510665e459eba73c18ef3e30e9b"
format_stamp: "Formatted at 2026-08-08 02:17:46 on dist-test-slave-1vmg"
server_key: "78ef11e09fe071d98a8d4e80699d4a6c"
server_key_iv: "8eba0da92dbdcdf4e947e0c401636792"
server_key_version: "encryptionkey@0"
I20260808 02:17:46.920405 25292 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260808 02:17:46.923468 25319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:46.925096 25292 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:17:46.925269 25292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "a2413510665e459eba73c18ef3e30e9b"
format_stamp: "Formatted at 2026-08-08 02:17:46 on dist-test-slave-1vmg"
server_key: "78ef11e09fe071d98a8d4e80699d4a6c"
server_key_iv: "8eba0da92dbdcdf4e947e0c401636792"
server_key_version: "encryptionkey@0"
I20260808 02:17:46.925407 25292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:46.957046 25292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:46.960927 25292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:46.961154 25292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:46.961854 25292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:17:46.962855 25292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:17:46.962949 25292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:46.963024 25292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:17:46.963061 25292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:46.972792 25292 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:44371
I20260808 02:17:46.972831 25489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:44371 every 8 connection(s)
I20260808 02:17:46.973798 25292 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:17:46.977048 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 25292
I20260808 02:17:46.977203 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:17:46.977487 24895 external_mini_cluster.cc:1428] Setting key 52c53bcab5ca5bf3a0a764aa43b76046
Aug 08 02:17:46 dist-test-slave-1vmg krb5kdc[24907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155466, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.79.195@KRBTEST.COM; defaulting to no policy
I20260808 02:17:46.988513 25030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:46.975956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:48793 (local address 127.24.79.254:46251)
0808 02:17:46.976133 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:46.976138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:46.977093 (+ 955us) server_negotiation.cc:407] Connection header received
0808 02:17:46.978557 (+ 1464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:46.978561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:46.978616 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:46.978690 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:46.980032 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:46.980567 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:46.981779 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:46.982005 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:46.982097 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:46.984726 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:46.984730 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:46.984732 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:46.984762 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:46.986173 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:46.986714 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:46.986718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:46.986720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:46.986809 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:46.987218 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:46.987222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:46.987223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:46.987469 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:46.987564 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:46.988178 (+ 614us) server_negotiation.cc:299] Negotiation successful
0808 02:17:46.988338 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:17:46.989460 25492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:46.976387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:46251 (local address 127.24.79.194:48793)
0808 02:17:46.976932 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:46.976981 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:46.978343 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:46.978889 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:46.978896 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:46.979329 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:46.979875 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:46.979892 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:46.980696 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:46.980699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:46.981643 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:46.981655 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:46.982160 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:46.982164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:46.982351 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:46.983004 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:46.983023 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:46.984538 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:46.986299 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:46.986306 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:46.986326 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:46.986608 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:46.986900 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:46.986903 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:46.986906 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:46.987110 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:46.987583 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:46.987591 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:46.987904 (+ 313us) client_negotiation.cc:769] Sending connection context
0808 02:17:46.988090 (+ 186us) client_negotiation.cc:240] Negotiation successful
0808 02:17:46.988492 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":122,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.195@KRBTEST.COM" created.
I20260808 02:17:46.990970 25490 heartbeater.cc:347] Connected to a master server at 127.24.79.254:46251
I20260808 02:17:46.991272 25490 heartbeater.cc:464] Registering TS with master...
I20260808 02:17:46.991855 25490 heartbeater.cc:511] Master 127.24.79.254:46251 requested a full tablet report, sending...
I20260808 02:17:46.993206 24975 ts_manager.cc:194] Registered new tserver with Master: a2413510665e459eba73c18ef3e30e9b (127.24.79.194:44371)
I20260808 02:17:46.994015 24975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.194@KRBTEST.COM'} at 127.24.79.194:48793
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
I20260808 02:17:47.011886 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:46251
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:43297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:47.137017 25496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:47.137344 25496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:47.137393 25496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:47.141829 25496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:47.141932 25496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:47.142002 25496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:17:47.148017 25496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:43297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:46251
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:47.149263 25496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:47.150421 25496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:47.161038 25508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:47.161031 25502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:47.161130 25503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:47.161485 25496 server_base.cc:1034] running on GCE node
I20260808 02:17:47.161928 25496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:47.162639 25496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:47.163815 25496 hybrid_clock.cc:648] HybridClock initialized: now 1786155467163779 us; error 51 us; skew 500 ppm
Aug 08 02:17:47 dist-test-slave-1vmg krb5kdc[24907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155467, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:47.166975 25496 init.cc:377] Logged in from keytab as kudu/127.24.79.195@KRBTEST.COM (short username kudu)
I20260808 02:17:47.168090 25496 webserver.cc:460] Webserver started at http://127.24.79.195:39091/ using document root <none> and password file <none>
I20260808 02:17:47.168704 25496 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:47.168784 25496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:47.169039 25496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:47.171433 25496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "fc2dca12547046109be6e04bedac2fe7"
format_stamp: "Formatted at 2026-08-08 02:17:47 on dist-test-slave-1vmg"
server_key: "5cbd9949949a35937194f558adbb2df4"
server_key_iv: "cc200563eac6eb6bc55281599a6c23c5"
server_key_version: "encryptionkey@0"
I20260808 02:17:47.172079 25496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "fc2dca12547046109be6e04bedac2fe7"
format_stamp: "Formatted at 2026-08-08 02:17:47 on dist-test-slave-1vmg"
server_key: "5cbd9949949a35937194f558adbb2df4"
server_key_iv: "cc200563eac6eb6bc55281599a6c23c5"
server_key_version: "encryptionkey@0"
I20260808 02:17:47.176478 25496 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:17:47.179373 25518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:47.180845 25496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 02:17:47.180989 25496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "fc2dca12547046109be6e04bedac2fe7"
format_stamp: "Formatted at 2026-08-08 02:17:47 on dist-test-slave-1vmg"
server_key: "5cbd9949949a35937194f558adbb2df4"
server_key_iv: "cc200563eac6eb6bc55281599a6c23c5"
server_key_version: "encryptionkey@0"
I20260808 02:17:47.181113 25496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:47.198774 25496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:47.203091 25496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:47.203325 25496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:47.204052 25496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:17:47.205051 25496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:17:47.205106 25496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:47.205168 25496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:17:47.205199 25496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:47.216655 25496 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:34833
I20260808 02:17:47.216698 25693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:34833 every 8 connection(s)
I20260808 02:17:47.217689 25496 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:17:47.221150 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 25496
I20260808 02:17:47.221295 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:17:47.221647 24895 external_mini_cluster.cc:1428] Setting key 7697b363beb01fb95bbedf72879107de
Aug 08 02:17:47 dist-test-slave-1vmg krb5kdc[24907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155467, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:17:47.233940 25030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:47.220081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:58201 (local address 127.24.79.254:46251)
0808 02:17:47.220340 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:47.220346 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:47.221475 (+ 1129us) server_negotiation.cc:407] Connection header received
0808 02:17:47.223088 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:47.223091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:47.223153 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:47.223230 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:47.224964 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:47.225565 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:47.226894 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:47.227282 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:47.227389 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:47.230350 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:47.230354 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:47.230356 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:47.230387 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:47.231789 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:47.232370 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:47.232373 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:47.232374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:47.232452 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:47.232777 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:47.232779 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:47.232780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:47.232964 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:47.233037 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:47.233687 (+ 650us) server_negotiation.cc:299] Negotiation successful
0808 02:17:47.233811 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260808 02:17:47.234879 25696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:47.220523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:46251 (local address 127.24.79.195:58201)
0808 02:17:47.221225 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:47.221302 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:47.222806 (+ 1504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:47.223440 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:47.223448 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:47.223953 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:47.224798 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:47.224825 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:47.225707 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:47.225710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:47.226733 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:47.226745 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:47.227489 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:47.227493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:47.227688 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:47.228418 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:47.228439 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:47.230116 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:47.231932 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:47.231938 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:47.231955 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:47.232268 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:47.232518 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:47.232520 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:47.232522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:47.232708 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:47.233096 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:47.233105 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:47.233455 (+ 350us) client_negotiation.cc:769] Sending connection context
0808 02:17:47.233647 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 02:17:47.233957 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":131,"threads_started":1}
I20260808 02:17:47.236270 25694 heartbeater.cc:347] Connected to a master server at 127.24.79.254:46251
I20260808 02:17:47.236555 25694 heartbeater.cc:464] Registering TS with master...
I20260808 02:17:47.237172 25694 heartbeater.cc:511] Master 127.24.79.254:46251 requested a full tablet report, sending...
I20260808 02:17:47.238349 24975 ts_manager.cc:194] Registered new tserver with Master: fc2dca12547046109be6e04bedac2fe7 (127.24.79.195:34833)
I20260808 02:17:47.239004 24975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.195@KRBTEST.COM'} at 127.24.79.195:58201
I20260808 02:17:47.249838 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 02:17:47 dist-test-slave-1vmg krb5kdc[24907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155467, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:17:47 dist-test-slave-1vmg krb5kdc[24907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155467, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.79.193@KRBTEST.COM
I20260808 02:17:47.280721 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:47.270343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55912 (local address 127.24.79.193:32957)
0808 02:17:47.270670 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:47.270678 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:47.270705 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:17:47.270761 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:47.270766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:47.271010 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:47.271144 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:47.271919 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:47.272748 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:47.273652 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:47.274072 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:47.274175 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:47.276911 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:47.276926 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:47.276943 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:47.276976 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:47.278629 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:47.279202 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:47.279209 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:47.279210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:47.279317 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:47.279663 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:47.279665 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:47.279666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:47.279977 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:47.280150 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:47.280427 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 02:17:47.280576 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":120,"threads_started":1}
W20260808 02:17:47.282070 25138 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55912
Aug 08 02:17:47 dist-test-slave-1vmg krb5kdc[24907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 02:17:47 dist-test-slave-1vmg krb5kdc[24907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.193@KRBTEST.COM
I20260808 02:17:47.312670 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:47.302758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55922 (local address 127.24.79.193:32957)
0808 02:17:47.302948 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:47.302954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:47.303039 (+ 85us) server_negotiation.cc:407] Connection header received
0808 02:17:47.303191 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:47.303194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:47.303251 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:47.303324 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:47.304143 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:47.304669 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:47.305528 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:47.305768 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:47.305864 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:47.308854 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:47.308858 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:47.308860 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:47.308905 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:47.310499 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:47.311061 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:47.311065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:47.311066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:47.311184 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:47.311584 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:47.311588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:47.311589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:47.311818 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:47.311944 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:47.312395 (+ 451us) server_negotiation.cc:299] Negotiation successful
0808 02:17:47.312524 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 02:17:47.314246 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 25066
I20260808 02:17:47.327773 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 25292
I20260808 02:17:47.340452 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 25496
I20260808 02:17:47.367017 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 24923
2026-08-08T02:17:47Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2696 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:47 dist-test-slave-1vmg krb5kdc[25713](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:47 dist-test-slave-1vmg krb5kdc[25713](info): set up 2 sockets
Aug 08 02:17:47 dist-test-slave-1vmg krb5kdc[25713](info): commencing operation
krb5kdc: starting...
W20260808 02:17:48.513403 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:17:48 dist-test-slave-1vmg krb5kdc[25713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:17:48.653951 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:37063
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:40031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:37063
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:48.778146 25736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:48.778486 25736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:48.778549 25736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:48.782840 25736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:48.782976 25736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:48.783010 25736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:48.783039 25736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:48.783069 25736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:48.788846 25736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:37063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:37063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:48.790094 25736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:48.791250 25736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:48.800689 25748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:48.800690 25745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:48.800689 25750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:48.801065 25736 server_base.cc:1034] running on GCE node
I20260808 02:17:48.801838 25736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:48.803221 25736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:48.804427 25736 hybrid_clock.cc:648] HybridClock initialized: now 1786155468804397 us; error 50 us; skew 500 ppm
Aug 08 02:17:48 dist-test-slave-1vmg krb5kdc[25713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155468, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:48.807662 25736 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:17:48.808688 25736 webserver.cc:460] Webserver started at http://127.24.79.254:36189/ using document root <none> and password file <none>
I20260808 02:17:48.809302 25736 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:48.809378 25736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:48.809614 25736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:48.812045 25736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "f7c0acfa6cdb4abaa07c70ffeb3ce892"
format_stamp: "Formatted at 2026-08-08 02:17:48 on dist-test-slave-1vmg"
server_key: "49b65867b62b8e8e1ed3b7dc87fcc8a0"
server_key_iv: "309608a4b69348d6302ad2d85190545e"
server_key_version: "encryptionkey@0"
I20260808 02:17:48.812701 25736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "f7c0acfa6cdb4abaa07c70ffeb3ce892"
format_stamp: "Formatted at 2026-08-08 02:17:48 on dist-test-slave-1vmg"
server_key: "49b65867b62b8e8e1ed3b7dc87fcc8a0"
server_key_iv: "309608a4b69348d6302ad2d85190545e"
server_key_version: "encryptionkey@0"
I20260808 02:17:48.817097 25736 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:17:48.819999 25760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:48.821502 25736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:17:48.821714 25736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "f7c0acfa6cdb4abaa07c70ffeb3ce892"
format_stamp: "Formatted at 2026-08-08 02:17:48 on dist-test-slave-1vmg"
server_key: "49b65867b62b8e8e1ed3b7dc87fcc8a0"
server_key_iv: "309608a4b69348d6302ad2d85190545e"
server_key_version: "encryptionkey@0"
I20260808 02:17:48.821851 25736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:48.836894 25736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:48.840513 25736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:48.840736 25736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:48.848259 25736 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:37063
I20260808 02:17:48.848342 25827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:37063 every 8 connection(s)
I20260808 02:17:48.849387 25736 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:48.852794 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 25736
I20260808 02:17:48.852927 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:17:48.852993 25829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:48.853220 24895 external_mini_cluster.cc:1428] Setting key 639c724d9c01a4a434f99df6add6e28a
I20260808 02:17:48.860560 25829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7c0acfa6cdb4abaa07c70ffeb3ce892: Bootstrap starting.
Aug 08 02:17:48 dist-test-slave-1vmg krb5kdc[25713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:17:48.863164 25829 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7c0acfa6cdb4abaa07c70ffeb3ce892: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:48.864065 25829 log.cc:824] T 00000000000000000000000000000000 P f7c0acfa6cdb4abaa07c70ffeb3ce892: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:48.866410 25829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7c0acfa6cdb4abaa07c70ffeb3ce892: No bootstrap required, opened a new log
I20260808 02:17:48.868867 25834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:48.854947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44914 (local address 127.24.79.254:37063)
0808 02:17:48.855518 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:48.855535 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:48.855578 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:17:48.856267 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:48.856287 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:48.856670 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:48.857023 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:48.857962 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:48.859127 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:48.860064 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:48.860584 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:48.860749 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:48.863367 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:48.863373 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:48.863391 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:48.863426 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:48.865511 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:48.866000 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:48.866006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:48.866012 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:48.866146 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:48.866477 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:48.866480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:48.866481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:48.867014 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:48.867223 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:48.867523 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 02:17:48.867850 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":187,"threads_started":1}
I20260808 02:17:48.871146 25829 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7c0acfa6cdb4abaa07c70ffeb3ce892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c0acfa6cdb4abaa07c70ffeb3ce892" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37063 } }
I20260808 02:17:48.871436 25829 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7c0acfa6cdb4abaa07c70ffeb3ce892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:48.871502 25829 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7c0acfa6cdb4abaa07c70ffeb3ce892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7c0acfa6cdb4abaa07c70ffeb3ce892, State: Initialized, Role: FOLLOWER
I20260808 02:17:48.872042 25829 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7c0acfa6cdb4abaa07c70ffeb3ce892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c0acfa6cdb4abaa07c70ffeb3ce892" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37063 } }
I20260808 02:17:48.872215 25829 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7c0acfa6cdb4abaa07c70ffeb3ce892 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:48.872287 25829 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7c0acfa6cdb4abaa07c70ffeb3ce892 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:48.872398 25829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7c0acfa6cdb4abaa07c70ffeb3ce892 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:48.873471 25829 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7c0acfa6cdb4abaa07c70ffeb3ce892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c0acfa6cdb4abaa07c70ffeb3ce892" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37063 } }
I20260808 02:17:48.873873 25829 leader_election.cc:302] T 00000000000000000000000000000000 P f7c0acfa6cdb4abaa07c70ffeb3ce892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7c0acfa6cdb4abaa07c70ffeb3ce892; no voters:
I20260808 02:17:48.874254 25829 leader_election.cc:288] T 00000000000000000000000000000000 P f7c0acfa6cdb4abaa07c70ffeb3ce892 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:48.874408 25837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7c0acfa6cdb4abaa07c70ffeb3ce892 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:48.874653 25837 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7c0acfa6cdb4abaa07c70ffeb3ce892 [term 1 LEADER]: Becoming Leader. State: Replica: f7c0acfa6cdb4abaa07c70ffeb3ce892, State: Running, Role: LEADER
I20260808 02:17:48.875031 25837 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7c0acfa6cdb4abaa07c70ffeb3ce892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c0acfa6cdb4abaa07c70ffeb3ce892" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37063 } }
I20260808 02:17:48.875718 25829 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7c0acfa6cdb4abaa07c70ffeb3ce892 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:48.876995 25839 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7c0acfa6cdb4abaa07c70ffeb3ce892 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7c0acfa6cdb4abaa07c70ffeb3ce892. Latest consensus state: current_term: 1 leader_uuid: "f7c0acfa6cdb4abaa07c70ffeb3ce892" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c0acfa6cdb4abaa07c70ffeb3ce892" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37063 } } }
I20260808 02:17:48.877017 25838 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7c0acfa6cdb4abaa07c70ffeb3ce892 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7c0acfa6cdb4abaa07c70ffeb3ce892" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c0acfa6cdb4abaa07c70ffeb3ce892" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37063 } } }
I20260808 02:17:48.877171 25839 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7c0acfa6cdb4abaa07c70ffeb3ce892 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:48.877170 25838 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7c0acfa6cdb4abaa07c70ffeb3ce892 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:48.877614 25848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:48.881093 25848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:48.888724 25848 catalog_manager.cc:1347] Generated new cluster ID: 6489a7f441e441ffad5265ffb9c95f0f
I20260808 02:17:48.888832 25848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:48.897166 25848 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:48.898357 25848 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:48.905196 25848 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7c0acfa6cdb4abaa07c70ffeb3ce892: Generated new TSK 0
I20260808 02:17:48.906189 25848 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.79.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
I20260808 02:17:48.947026 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:37063
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:40031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:49.073706 25866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:49.074043 25866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:49.074121 25866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:49.078532 25866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:49.078625 25866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:49.078713 25866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:17:49.084506 25866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:37063
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:49.085739 25866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:49.086839 25866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:49.097111 25877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:49.097146 25872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:49.097375 25866 server_base.cc:1034] running on GCE node
W20260808 02:17:49.097294 25873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:49.097986 25866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:49.098764 25866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:49.099954 25866 hybrid_clock.cc:648] HybridClock initialized: now 1786155469099897 us; error 76 us; skew 500 ppm
Aug 08 02:17:49 dist-test-slave-1vmg krb5kdc[25713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155469, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:49.103047 25866 init.cc:377] Logged in from keytab as kudu/127.24.79.193@KRBTEST.COM (short username kudu)
I20260808 02:17:49.104061 25866 webserver.cc:460] Webserver started at http://127.24.79.193:46149/ using document root <none> and password file <none>
I20260808 02:17:49.104646 25866 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:49.104710 25866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:49.104905 25866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:49.107255 25866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "4ddda691326c441eb58c3f1045a2e67d"
format_stamp: "Formatted at 2026-08-08 02:17:49 on dist-test-slave-1vmg"
server_key: "346733402a88c9f0826def629056991b"
server_key_iv: "82974820732b94c091d33997a2a66f77"
server_key_version: "encryptionkey@0"
I20260808 02:17:49.107904 25866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "4ddda691326c441eb58c3f1045a2e67d"
format_stamp: "Formatted at 2026-08-08 02:17:49 on dist-test-slave-1vmg"
server_key: "346733402a88c9f0826def629056991b"
server_key_iv: "82974820732b94c091d33997a2a66f77"
server_key_version: "encryptionkey@0"
I20260808 02:17:49.112133 25866 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:17:49.115036 25885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:49.116498 25866 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:17:49.116631 25866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "4ddda691326c441eb58c3f1045a2e67d"
format_stamp: "Formatted at 2026-08-08 02:17:49 on dist-test-slave-1vmg"
server_key: "346733402a88c9f0826def629056991b"
server_key_iv: "82974820732b94c091d33997a2a66f77"
server_key_version: "encryptionkey@0"
I20260808 02:17:49.116741 25866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:49.137437 25866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:49.141237 25866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:49.141443 25866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:49.142096 25866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:17:49.143076 25866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:17:49.143138 25866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:49.143183 25866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:17:49.143204 25866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:49.152884 25866 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:35943
I20260808 02:17:49.152967 26062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:35943 every 8 connection(s)
I20260808 02:17:49.154018 25866 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:17:49.156054 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 25866
I20260808 02:17:49.156176 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:17:49.156502 24895 external_mini_cluster.cc:1428] Setting key 1e4d196a00a2e3daa847c548ba7cb331
Aug 08 02:17:49 dist-test-slave-1vmg krb5kdc[25713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155469, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.79.194@KRBTEST.COM; defaulting to no policy
I20260808 02:17:49.171101 25834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:49.156385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:41425 (local address 127.24.79.254:37063)
0808 02:17:49.156568 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:49.156574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:49.157627 (+ 1053us) server_negotiation.cc:407] Connection header received
0808 02:17:49.158843 (+ 1216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:49.158846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:49.158896 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:49.159020 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:49.160825 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.161715 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.163052 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.163409 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.163553 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:49.166681 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:49.166686 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:49.166689 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:49.166725 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:49.168401 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:49.168997 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:49.169006 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:49.169008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:49.169212 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:49.169669 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:49.169672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:49.169674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:49.169915 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:49.170012 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:49.170710 (+ 698us) server_negotiation.cc:299] Negotiation successful
0808 02:17:49.170851 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.194@KRBTEST.COM" created.
I20260808 02:17:49.172215 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:49.156890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:37063 (local address 127.24.79.193:41425)
0808 02:17:49.157470 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:49.157523 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:49.158564 (+ 1041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:49.159251 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:49.159261 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:49.159831 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:49.160605 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:49.160632 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.161871 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.161879 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:49.162854 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:49.162866 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.163628 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.163632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:49.163908 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:49.164645 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:49.164665 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:49.166443 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:49.168537 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:49.168545 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:49.168562 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:49.168870 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:49.169328 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:49.169331 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:49.169333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:49.169535 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:49.170052 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:49.170059 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:49.170385 (+ 326us) client_negotiation.cc:769] Sending connection context
0808 02:17:49.170615 (+ 230us) client_negotiation.cc:240] Negotiation successful
0808 02:17:49.171039 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":146,"threads_started":1}
I20260808 02:17:49.173712 26063 heartbeater.cc:347] Connected to a master server at 127.24.79.254:37063
I20260808 02:17:49.173995 26063 heartbeater.cc:464] Registering TS with master...
I20260808 02:17:49.174572 26063 heartbeater.cc:511] Master 127.24.79.254:37063 requested a full tablet report, sending...
I20260808 02:17:49.176450 25782 ts_manager.cc:194] Registered new tserver with Master: 4ddda691326c441eb58c3f1045a2e67d (127.24.79.193:35943)
I20260808 02:17:49.177739 25782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.193@KRBTEST.COM'} at 127.24.79.193:41425
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
I20260808 02:17:49.191143 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:37063
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:40031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:49.313187 26068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:49.313517 26068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:49.313566 26068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:49.317821 26068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:49.317917 26068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:49.317981 26068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:17:49.324038 26068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:37063
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:49.325407 26068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:49.326759 26068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:17:49.339192 26068 server_base.cc:1034] running on GCE node
W20260808 02:17:49.339124 26077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:49.339126 26082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:49.339126 26079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:49.339995 26068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:49.340739 26068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:49.341913 26068 hybrid_clock.cc:648] HybridClock initialized: now 1786155469341882 us; error 49 us; skew 500 ppm
Aug 08 02:17:49 dist-test-slave-1vmg krb5kdc[25713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155469, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:49.345292 26068 init.cc:377] Logged in from keytab as kudu/127.24.79.194@KRBTEST.COM (short username kudu)
I20260808 02:17:49.346398 26068 webserver.cc:460] Webserver started at http://127.24.79.194:43969/ using document root <none> and password file <none>
I20260808 02:17:49.347076 26068 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:49.347165 26068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:49.347426 26068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:49.349797 26068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "1799c5cbe3ea4ef7821fb0e76a4d5a5e"
format_stamp: "Formatted at 2026-08-08 02:17:49 on dist-test-slave-1vmg"
server_key: "e49a371b547e924bd044ea2152c3257d"
server_key_iv: "6383e985fefe7452aef9115b21d11e2d"
server_key_version: "encryptionkey@0"
I20260808 02:17:49.350445 26068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "1799c5cbe3ea4ef7821fb0e76a4d5a5e"
format_stamp: "Formatted at 2026-08-08 02:17:49 on dist-test-slave-1vmg"
server_key: "e49a371b547e924bd044ea2152c3257d"
server_key_iv: "6383e985fefe7452aef9115b21d11e2d"
server_key_version: "encryptionkey@0"
I20260808 02:17:49.354898 26068 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 02:17:49.357841 26090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:49.359503 26068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 02:17:49.359650 26068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "1799c5cbe3ea4ef7821fb0e76a4d5a5e"
format_stamp: "Formatted at 2026-08-08 02:17:49 on dist-test-slave-1vmg"
server_key: "e49a371b547e924bd044ea2152c3257d"
server_key_iv: "6383e985fefe7452aef9115b21d11e2d"
server_key_version: "encryptionkey@0"
I20260808 02:17:49.359797 26068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:49.385572 26068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:49.389139 26068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:49.389350 26068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:49.389994 26068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:17:49.390950 26068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:17:49.391007 26068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:49.391065 26068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:17:49.391091 26068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:49.400215 26068 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:38219
I20260808 02:17:49.400236 26261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:38219 every 8 connection(s)
I20260808 02:17:49.401189 26068 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:17:49.409938 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 26068
I20260808 02:17:49.410035 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:17:49.410302 24895 external_mini_cluster.cc:1428] Setting key ceb01d317e54b861fa6ec00b78e90f57
Aug 08 02:17:49 dist-test-slave-1vmg krb5kdc[25713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155469, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:17:49.415884 25834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:49.403094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:56953 (local address 127.24.79.254:37063)
0808 02:17:49.403233 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:49.403236 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:49.404156 (+ 920us) server_negotiation.cc:407] Connection header received
0808 02:17:49.405153 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:49.405156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:49.405211 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:49.405288 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:49.406564 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.407183 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.408453 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.408694 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.408813 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:49.411715 (+ 2902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:49.411719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:49.411722 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:49.411753 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:49.413363 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:49.413934 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:49.413938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:49.413940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:49.414043 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:49.414502 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:49.414505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:49.414507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:49.414734 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:49.414817 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:49.415584 (+ 767us) server_negotiation.cc:299] Negotiation successful
0808 02:17:49.415752 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 02:17:49.417065 26266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:49.403482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:37063 (local address 127.24.79.194:56953)
0808 02:17:49.403965 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:49.404012 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:49.404927 (+ 915us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:49.405476 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:49.405484 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:49.405872 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:49.406416 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:49.406430 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.407319 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.407322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:49.408293 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:49.408309 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.408884 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.408887 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:49.409088 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:49.409783 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:49.409802 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:49.411504 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:49.413500 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:49.413506 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:49.413521 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:49.413832 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:49.414148 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:49.414151 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:49.414154 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:49.414393 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:49.414838 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:49.414845 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:49.415267 (+ 422us) client_negotiation.cc:769] Sending connection context
0808 02:17:49.415516 (+ 249us) client_negotiation.cc:240] Negotiation successful
0808 02:17:49.415931 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":113,"threads_started":1}
I20260808 02:17:49.418540 26263 heartbeater.cc:347] Connected to a master server at 127.24.79.254:37063
I20260808 02:17:49.418869 26263 heartbeater.cc:464] Registering TS with master...
I20260808 02:17:49.419631 26263 heartbeater.cc:511] Master 127.24.79.254:37063 requested a full tablet report, sending...
I20260808 02:17:49.421145 25782 ts_manager.cc:194] Registered new tserver with Master: 1799c5cbe3ea4ef7821fb0e76a4d5a5e (127.24.79.194:38219)
I20260808 02:17:49.422047 25782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.194@KRBTEST.COM'} at 127.24.79.194:56953
WARNING: no policy specified for kudu/127.24.79.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
I20260808 02:17:49.442854 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:37063
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:40031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:49.566394 26269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:49.566705 26269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:49.566751 26269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:49.570863 26269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:49.570968 26269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:49.571033 26269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:17:49.576676 26269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:37063
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:49.577876 26269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:49.578881 26269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:49.589146 26275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:49.589139 26274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:49.589413 26269 server_base.cc:1034] running on GCE node
W20260808 02:17:49.589216 26277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:49.590014 26269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:49.590740 26269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:49.591935 26269 hybrid_clock.cc:648] HybridClock initialized: now 1786155469591909 us; error 55 us; skew 500 ppm
Aug 08 02:17:49 dist-test-slave-1vmg krb5kdc[25713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155469, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:49.595214 26269 init.cc:377] Logged in from keytab as kudu/127.24.79.195@KRBTEST.COM (short username kudu)
I20260808 02:17:49.596271 26269 webserver.cc:460] Webserver started at http://127.24.79.195:36807/ using document root <none> and password file <none>
I20260808 02:17:49.596879 26269 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:49.596956 26269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:49.597184 26269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:49.599593 26269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "ec9bdcb6eb274a3d81481d180f0d92da"
format_stamp: "Formatted at 2026-08-08 02:17:49 on dist-test-slave-1vmg"
server_key: "bc52b009bb8966f0f2e854e37f427765"
server_key_iv: "23991b31c15eec82e6353fa8d6dc2e60"
server_key_version: "encryptionkey@0"
I20260808 02:17:49.600214 26269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "ec9bdcb6eb274a3d81481d180f0d92da"
format_stamp: "Formatted at 2026-08-08 02:17:49 on dist-test-slave-1vmg"
server_key: "bc52b009bb8966f0f2e854e37f427765"
server_key_iv: "23991b31c15eec82e6353fa8d6dc2e60"
server_key_version: "encryptionkey@0"
I20260808 02:17:49.604240 26269 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:17:49.606977 26290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:49.608436 26269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:17:49.608568 26269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "ec9bdcb6eb274a3d81481d180f0d92da"
format_stamp: "Formatted at 2026-08-08 02:17:49 on dist-test-slave-1vmg"
server_key: "bc52b009bb8966f0f2e854e37f427765"
server_key_iv: "23991b31c15eec82e6353fa8d6dc2e60"
server_key_version: "encryptionkey@0"
I20260808 02:17:49.608721 26269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:49.627604 26269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:49.631007 26269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:49.631184 26269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:49.631875 26269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:17:49.632776 26269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:17:49.632824 26269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:49.632867 26269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:17:49.632889 26269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:49.643018 26269 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:36693
I20260808 02:17:49.643064 26463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:36693 every 8 connection(s)
I20260808 02:17:49.644058 26269 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:17:49.652019 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 26269
I20260808 02:17:49.652159 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:17:49.652479 24895 external_mini_cluster.cc:1428] Setting key 96789a2391a34cdad8c27ec955685d4f
Aug 08 02:17:49 dist-test-slave-1vmg krb5kdc[25713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155469, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:17:49.659204 25834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:49.646155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:57899 (local address 127.24.79.254:37063)
0808 02:17:49.646338 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:49.646342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:49.647639 (+ 1297us) server_negotiation.cc:407] Connection header received
0808 02:17:49.648536 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:49.648539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:49.648588 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:49.648672 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:49.649979 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.650537 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.651892 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.652259 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.652383 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:49.655203 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:49.655206 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:49.655209 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:49.655244 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:49.656643 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:49.657370 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:49.657374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:49.657375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:49.657474 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:49.657870 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:49.657873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:49.657874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:49.658091 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:49.658184 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:49.658826 (+ 642us) server_negotiation.cc:299] Negotiation successful
0808 02:17:49.659026 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 02:17:49.660169 26466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:49.646547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:37063 (local address 127.24.79.195:57899)
0808 02:17:49.647451 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:49.647505 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:49.648345 (+ 840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:49.648843 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:49.648851 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:49.649256 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:49.649835 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:49.649851 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.650693 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.650697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:49.651730 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:49.651743 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.652428 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.652431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:49.652645 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:49.653317 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:49.653336 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:49.655027 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:49.656856 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:49.656865 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:49.656888 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:49.657230 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:49.657571 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:49.657575 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:49.657577 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:49.657758 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:49.658206 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:49.658215 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:49.658538 (+ 323us) client_negotiation.cc:769] Sending connection context
0808 02:17:49.658731 (+ 193us) client_negotiation.cc:240] Negotiation successful
0808 02:17:49.659191 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":144,"threads_started":1}
I20260808 02:17:49.661808 26464 heartbeater.cc:347] Connected to a master server at 127.24.79.254:37063
I20260808 02:17:49.662091 26464 heartbeater.cc:464] Registering TS with master...
I20260808 02:17:49.662653 26464 heartbeater.cc:511] Master 127.24.79.254:37063 requested a full tablet report, sending...
I20260808 02:17:49.664057 25782 ts_manager.cc:194] Registered new tserver with Master: ec9bdcb6eb274a3d81481d180f0d92da (127.24.79.195:36693)
I20260808 02:17:49.664752 25782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.195@KRBTEST.COM'} at 127.24.79.195:57899
I20260808 02:17:49.666827 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:17:49.680138 25834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:49.671639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44926 (local address 127.24.79.254:37063)
0808 02:17:49.671844 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:49.671849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:49.671992 (+ 143us) server_negotiation.cc:407] Connection header received
0808 02:17:49.672155 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:49.672160 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:49.672219 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:49.672302 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:49.673139 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.673684 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.674565 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.674805 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.674909 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:49.676509 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:49.676513 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:49.676516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:49.676557 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:49.678254 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:49.678732 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:49.678735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:49.678737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:49.678831 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:49.679136 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:49.679140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:49.679141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:49.679407 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:49.679538 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:49.679824 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 02:17:49.679970 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 02:17:49.687790 25782 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44926:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:17:49.691519 25782 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:17:49.710099 26487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:49.704547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40960 (local address 127.24.79.194:38219)
0808 02:17:49.705088 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:49.705100 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:49.705123 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:17:49.705203 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:49.705208 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:49.705395 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:49.705537 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:49.705713 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.706523 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.708369 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.709221 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.709328 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:49.709590 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:49.709818 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 02:17:49.709918 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":124,"threads_started":1}
I20260808 02:17:49.710110 26481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:49.703506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:38219 (local address 127.0.0.1:40960)
0808 02:17:49.704265 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:49.704280 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:49.704376 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:49.705513 (+ 1137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:49.705516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:49.705534 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:49.705627 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:49.705633 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.706663 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.706665 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:49.708250 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:49.708262 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.709452 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.709456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:49.709614 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:49.709660 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 02:17:49.709771 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 02:17:49.709915 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"mutex_wait_us":115,"thread_start_us":188,"threads_started":1}
I20260808 02:17:49.710436 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:49.703510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40044 (local address 127.24.79.195:36693)
0808 02:17:49.704215 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:49.704230 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:49.704257 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:17:49.704340 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:49.704346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:49.704572 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:49.704756 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:49.705619 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.706750 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.708415 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.709274 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.709385 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:49.709594 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:49.710202 (+ 608us) server_negotiation.cc:299] Negotiation successful
0808 02:17:49.710299 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":182,"threads_started":1}
I20260808 02:17:49.710130 26484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:49.704051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:35943 (local address 127.0.0.1:33012)
0808 02:17:49.704955 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:49.704969 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:49.705083 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:49.705707 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:49.705710 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:49.705729 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:49.705826 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:49.705831 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.707168 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.707171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:49.708566 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:49.708589 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.709779 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.709783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:49.709856 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:49.709876 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 02:17:49.709928 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 02:17:49.709980 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":844,"mutex_wait_us":167,"thread_start_us":115,"threads_started":1}
I20260808 02:17:49.710131 26482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:49.703352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:36693 (local address 127.0.0.1:40044)
0808 02:17:49.703980 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:49.704041 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:49.704204 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:49.705183 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:49.705189 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:49.705211 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:49.705380 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:49.705388 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.706890 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.706893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:49.708256 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:49.708263 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.709673 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.709677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:49.709778 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:49.709803 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 02:17:49.709871 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 02:17:49.709950 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":162,"thread_start_us":91,"threads_started":1}
I20260808 02:17:49.711510 26486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:49.704517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33012 (local address 127.24.79.193:35943)
0808 02:17:49.704921 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:49.704928 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:49.705067 (+ 139us) server_negotiation.cc:407] Connection header received
0808 02:17:49.705246 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:49.705251 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:49.705395 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:49.705537 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:49.705960 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.707053 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.708718 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.709499 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.709598 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:49.709724 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:49.710202 (+ 478us) server_negotiation.cc:299] Negotiation successful
0808 02:17:49.710299 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":88,"threads_started":1}
I20260808 02:17:49.714015 26366 tablet_service.cc:1467] Processing CreateTablet for tablet 4a42802275a84bb0962691d95fbfe7c5 (DEFAULT_TABLE table=test-table [id=bdc140c66bd74f6caaa1cc3aad5f65d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:49.714175 25960 tablet_service.cc:1467] Processing CreateTablet for tablet 4a42802275a84bb0962691d95fbfe7c5 (DEFAULT_TABLE table=test-table [id=bdc140c66bd74f6caaa1cc3aad5f65d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:49.715308 25960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a42802275a84bb0962691d95fbfe7c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:49.716786 26366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a42802275a84bb0962691d95fbfe7c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:49.716488 26169 tablet_service.cc:1467] Processing CreateTablet for tablet 4a42802275a84bb0962691d95fbfe7c5 (DEFAULT_TABLE table=test-table [id=bdc140c66bd74f6caaa1cc3aad5f65d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:49.717654 26169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a42802275a84bb0962691d95fbfe7c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:49.723750 26488 tablet_bootstrap.cc:492] T 4a42802275a84bb0962691d95fbfe7c5 P ec9bdcb6eb274a3d81481d180f0d92da: Bootstrap starting.
I20260808 02:17:49.725323 26492 tablet_bootstrap.cc:492] T 4a42802275a84bb0962691d95fbfe7c5 P 4ddda691326c441eb58c3f1045a2e67d: Bootstrap starting.
I20260808 02:17:49.726787 26488 tablet_bootstrap.cc:654] T 4a42802275a84bb0962691d95fbfe7c5 P ec9bdcb6eb274a3d81481d180f0d92da: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:49.727865 26492 tablet_bootstrap.cc:654] T 4a42802275a84bb0962691d95fbfe7c5 P 4ddda691326c441eb58c3f1045a2e67d: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:49.728595 26488 log.cc:824] T 4a42802275a84bb0962691d95fbfe7c5 P ec9bdcb6eb274a3d81481d180f0d92da: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:49.728639 26492 log.cc:824] T 4a42802275a84bb0962691d95fbfe7c5 P 4ddda691326c441eb58c3f1045a2e67d: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:49.729426 26493 tablet_bootstrap.cc:492] T 4a42802275a84bb0962691d95fbfe7c5 P 1799c5cbe3ea4ef7821fb0e76a4d5a5e: Bootstrap starting.
I20260808 02:17:49.730973 26488 tablet_bootstrap.cc:492] T 4a42802275a84bb0962691d95fbfe7c5 P ec9bdcb6eb274a3d81481d180f0d92da: No bootstrap required, opened a new log
I20260808 02:17:49.731217 26488 ts_tablet_manager.cc:1397] T 4a42802275a84bb0962691d95fbfe7c5 P ec9bdcb6eb274a3d81481d180f0d92da: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 02:17:49.731395 26492 tablet_bootstrap.cc:492] T 4a42802275a84bb0962691d95fbfe7c5 P 4ddda691326c441eb58c3f1045a2e67d: No bootstrap required, opened a new log
I20260808 02:17:49.731600 26492 ts_tablet_manager.cc:1397] T 4a42802275a84bb0962691d95fbfe7c5 P 4ddda691326c441eb58c3f1045a2e67d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 02:17:49.732321 26493 tablet_bootstrap.cc:654] T 4a42802275a84bb0962691d95fbfe7c5 P 1799c5cbe3ea4ef7821fb0e76a4d5a5e: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:49.733307 26493 log.cc:824] T 4a42802275a84bb0962691d95fbfe7c5 P 1799c5cbe3ea4ef7821fb0e76a4d5a5e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:49.734701 26488 raft_consensus.cc:348] T 4a42802275a84bb0962691d95fbfe7c5 P ec9bdcb6eb274a3d81481d180f0d92da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9bdcb6eb274a3d81481d180f0d92da" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 36693 } } peers { permanent_uuid: "1799c5cbe3ea4ef7821fb0e76a4d5a5e" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 38219 } } peers { permanent_uuid: "4ddda691326c441eb58c3f1045a2e67d" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 35943 } }
I20260808 02:17:49.735052 26488 raft_consensus.cc:374] T 4a42802275a84bb0962691d95fbfe7c5 P ec9bdcb6eb274a3d81481d180f0d92da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:49.735112 26488 raft_consensus.cc:729] T 4a42802275a84bb0962691d95fbfe7c5 P ec9bdcb6eb274a3d81481d180f0d92da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec9bdcb6eb274a3d81481d180f0d92da, State: Initialized, Role: FOLLOWER
I20260808 02:17:49.735648 26488 consensus_queue.cc:260] T 4a42802275a84bb0962691d95fbfe7c5 P ec9bdcb6eb274a3d81481d180f0d92da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9bdcb6eb274a3d81481d180f0d92da" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 36693 } } peers { permanent_uuid: "1799c5cbe3ea4ef7821fb0e76a4d5a5e" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 38219 } } peers { permanent_uuid: "4ddda691326c441eb58c3f1045a2e67d" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 35943 } }
I20260808 02:17:49.735976 26493 tablet_bootstrap.cc:492] T 4a42802275a84bb0962691d95fbfe7c5 P 1799c5cbe3ea4ef7821fb0e76a4d5a5e: No bootstrap required, opened a new log
I20260808 02:17:49.736196 26493 ts_tablet_manager.cc:1397] T 4a42802275a84bb0962691d95fbfe7c5 P 1799c5cbe3ea4ef7821fb0e76a4d5a5e: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 02:17:49.736120 26492 raft_consensus.cc:348] T 4a42802275a84bb0962691d95fbfe7c5 P 4ddda691326c441eb58c3f1045a2e67d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9bdcb6eb274a3d81481d180f0d92da" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 36693 } } peers { permanent_uuid: "1799c5cbe3ea4ef7821fb0e76a4d5a5e" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 38219 } } peers { permanent_uuid: "4ddda691326c441eb58c3f1045a2e67d" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 35943 } }
I20260808 02:17:49.736418 26492 raft_consensus.cc:374] T 4a42802275a84bb0962691d95fbfe7c5 P 4ddda691326c441eb58c3f1045a2e67d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:49.736480 26492 raft_consensus.cc:729] T 4a42802275a84bb0962691d95fbfe7c5 P 4ddda691326c441eb58c3f1045a2e67d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ddda691326c441eb58c3f1045a2e67d, State: Initialized, Role: FOLLOWER
I20260808 02:17:49.736781 26488 ts_tablet_manager.cc:1428] T 4a42802275a84bb0962691d95fbfe7c5 P ec9bdcb6eb274a3d81481d180f0d92da: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 02:17:49.736939 26492 consensus_queue.cc:260] T 4a42802275a84bb0962691d95fbfe7c5 P 4ddda691326c441eb58c3f1045a2e67d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9bdcb6eb274a3d81481d180f0d92da" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 36693 } } peers { permanent_uuid: "1799c5cbe3ea4ef7821fb0e76a4d5a5e" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 38219 } } peers { permanent_uuid: "4ddda691326c441eb58c3f1045a2e67d" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 35943 } }
I20260808 02:17:49.737298 26464 heartbeater.cc:503] Master 127.24.79.254:37063 was elected leader, sending a full tablet report...
I20260808 02:17:49.737649 26063 heartbeater.cc:503] Master 127.24.79.254:37063 was elected leader, sending a full tablet report...
I20260808 02:17:49.737991 26492 ts_tablet_manager.cc:1428] T 4a42802275a84bb0962691d95fbfe7c5 P 4ddda691326c441eb58c3f1045a2e67d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 02:17:49.740571 26493 raft_consensus.cc:348] T 4a42802275a84bb0962691d95fbfe7c5 P 1799c5cbe3ea4ef7821fb0e76a4d5a5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9bdcb6eb274a3d81481d180f0d92da" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 36693 } } peers { permanent_uuid: "1799c5cbe3ea4ef7821fb0e76a4d5a5e" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 38219 } } peers { permanent_uuid: "4ddda691326c441eb58c3f1045a2e67d" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 35943 } }
I20260808 02:17:49.740830 26493 raft_consensus.cc:374] T 4a42802275a84bb0962691d95fbfe7c5 P 1799c5cbe3ea4ef7821fb0e76a4d5a5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:49.740881 26493 raft_consensus.cc:729] T 4a42802275a84bb0962691d95fbfe7c5 P 1799c5cbe3ea4ef7821fb0e76a4d5a5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1799c5cbe3ea4ef7821fb0e76a4d5a5e, State: Initialized, Role: FOLLOWER
I20260808 02:17:49.741330 26493 consensus_queue.cc:260] T 4a42802275a84bb0962691d95fbfe7c5 P 1799c5cbe3ea4ef7821fb0e76a4d5a5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9bdcb6eb274a3d81481d180f0d92da" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 36693 } } peers { permanent_uuid: "1799c5cbe3ea4ef7821fb0e76a4d5a5e" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 38219 } } peers { permanent_uuid: "4ddda691326c441eb58c3f1045a2e67d" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 35943 } }
I20260808 02:17:49.742115 26263 heartbeater.cc:503] Master 127.24.79.254:37063 was elected leader, sending a full tablet report...
I20260808 02:17:49.742463 26493 ts_tablet_manager.cc:1428] T 4a42802275a84bb0962691d95fbfe7c5 P 1799c5cbe3ea4ef7821fb0e76a4d5a5e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 02:17:49.754691 26502 raft_consensus.cc:482] T 4a42802275a84bb0962691d95fbfe7c5 P 1799c5cbe3ea4ef7821fb0e76a4d5a5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:17:49.754961 26502 raft_consensus.cc:504] T 4a42802275a84bb0962691d95fbfe7c5 P 1799c5cbe3ea4ef7821fb0e76a4d5a5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9bdcb6eb274a3d81481d180f0d92da" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 36693 } } peers { permanent_uuid: "1799c5cbe3ea4ef7821fb0e76a4d5a5e" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 38219 } } peers { permanent_uuid: "4ddda691326c441eb58c3f1045a2e67d" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 35943 } }
I20260808 02:17:49.756269 26502 leader_election.cc:288] T 4a42802275a84bb0962691d95fbfe7c5 P 1799c5cbe3ea4ef7821fb0e76a4d5a5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec9bdcb6eb274a3d81481d180f0d92da (127.24.79.195:36693), 4ddda691326c441eb58c3f1045a2e67d (127.24.79.193:35943)
I20260808 02:17:49.760053 26486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:49.756828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:38217 (local address 127.24.79.193:35943)
0808 02:17:49.756960 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:49.756964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:49.756977 (+ 13us) server_negotiation.cc:407] Connection header received
0808 02:17:49.757028 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:49.757030 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:49.757069 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:49.757148 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:49.757421 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.757949 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.759059 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.759482 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.759597 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:49.759666 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:49.759843 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 02:17:49.759920 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 02:17:49.760080 26266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:49.756543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:35943 (local address 127.24.79.194:38217)
0808 02:17:49.756799 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:49.756815 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:49.756919 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:49.757179 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:49.757183 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:49.757202 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:49.757311 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:49.757317 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.758058 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.758062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:49.758896 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:49.758906 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.759597 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.759601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:49.759716 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:49.759735 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 02:17:49.759817 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 02:17:49.759899 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157}
I20260808 02:17:49.760123 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:49.756628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:40239 (local address 127.24.79.195:36693)
0808 02:17:49.756777 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:49.756782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:49.757093 (+ 311us) server_negotiation.cc:407] Connection header received
0808 02:17:49.757191 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:49.757194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:49.757241 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:49.757300 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:49.757554 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.758052 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.759145 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.759568 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.759678 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:49.759742 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:49.759930 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 02:17:49.760002 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:17:49.760135 26503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:49.756543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:36693 (local address 127.24.79.194:40239)
0808 02:17:49.757001 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:49.757013 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:49.757108 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:49.757319 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:49.757322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:49.757338 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:49.757456 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:49.757461 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.758152 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.758155 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:49.759021 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:49.759031 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.759681 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.759684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:49.759814 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:49.759837 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:17:49.759902 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 02:17:49.759972 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":35,"thread_start_us":185,"threads_started":1}
I20260808 02:17:49.760850 25989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a42802275a84bb0962691d95fbfe7c5" candidate_uuid: "1799c5cbe3ea4ef7821fb0e76a4d5a5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ddda691326c441eb58c3f1045a2e67d" is_pre_election: true
I20260808 02:17:49.761250 25989 raft_consensus.cc:2454] T 4a42802275a84bb0962691d95fbfe7c5 P 4ddda691326c441eb58c3f1045a2e67d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1799c5cbe3ea4ef7821fb0e76a4d5a5e in term 0.
I20260808 02:17:49.761767 26092 leader_election.cc:302] T 4a42802275a84bb0962691d95fbfe7c5 P 1799c5cbe3ea4ef7821fb0e76a4d5a5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1799c5cbe3ea4ef7821fb0e76a4d5a5e, 4ddda691326c441eb58c3f1045a2e67d; no voters:
I20260808 02:17:49.762069 26502 raft_consensus.cc:2781] T 4a42802275a84bb0962691d95fbfe7c5 P 1799c5cbe3ea4ef7821fb0e76a4d5a5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:17:49.762161 26502 raft_consensus.cc:482] T 4a42802275a84bb0962691d95fbfe7c5 P 1799c5cbe3ea4ef7821fb0e76a4d5a5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:17:49.762193 26502 raft_consensus.cc:3037] T 4a42802275a84bb0962691d95fbfe7c5 P 1799c5cbe3ea4ef7821fb0e76a4d5a5e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:49.762352 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a42802275a84bb0962691d95fbfe7c5" candidate_uuid: "1799c5cbe3ea4ef7821fb0e76a4d5a5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec9bdcb6eb274a3d81481d180f0d92da" is_pre_election: true
I20260808 02:17:49.762671 26399 raft_consensus.cc:2454] T 4a42802275a84bb0962691d95fbfe7c5 P ec9bdcb6eb274a3d81481d180f0d92da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1799c5cbe3ea4ef7821fb0e76a4d5a5e in term 0.
I20260808 02:17:49.763504 26502 raft_consensus.cc:504] T 4a42802275a84bb0962691d95fbfe7c5 P 1799c5cbe3ea4ef7821fb0e76a4d5a5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9bdcb6eb274a3d81481d180f0d92da" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 36693 } } peers { permanent_uuid: "1799c5cbe3ea4ef7821fb0e76a4d5a5e" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 38219 } } peers { permanent_uuid: "4ddda691326c441eb58c3f1045a2e67d" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 35943 } }
I20260808 02:17:49.763882 26502 leader_election.cc:288] T 4a42802275a84bb0962691d95fbfe7c5 P 1799c5cbe3ea4ef7821fb0e76a4d5a5e [CANDIDATE]: Term 1 election: Requested vote from peers ec9bdcb6eb274a3d81481d180f0d92da (127.24.79.195:36693), 4ddda691326c441eb58c3f1045a2e67d (127.24.79.193:35943)
I20260808 02:17:49.764314 26399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a42802275a84bb0962691d95fbfe7c5" candidate_uuid: "1799c5cbe3ea4ef7821fb0e76a4d5a5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec9bdcb6eb274a3d81481d180f0d92da"
I20260808 02:17:49.764350 25989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a42802275a84bb0962691d95fbfe7c5" candidate_uuid: "1799c5cbe3ea4ef7821fb0e76a4d5a5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ddda691326c441eb58c3f1045a2e67d"
I20260808 02:17:49.764462 26399 raft_consensus.cc:3037] T 4a42802275a84bb0962691d95fbfe7c5 P ec9bdcb6eb274a3d81481d180f0d92da [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:49.764461 25989 raft_consensus.cc:3037] T 4a42802275a84bb0962691d95fbfe7c5 P 4ddda691326c441eb58c3f1045a2e67d [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:49.765872 26399 raft_consensus.cc:2454] T 4a42802275a84bb0962691d95fbfe7c5 P ec9bdcb6eb274a3d81481d180f0d92da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1799c5cbe3ea4ef7821fb0e76a4d5a5e in term 1.
I20260808 02:17:49.765872 25989 raft_consensus.cc:2454] T 4a42802275a84bb0962691d95fbfe7c5 P 4ddda691326c441eb58c3f1045a2e67d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1799c5cbe3ea4ef7821fb0e76a4d5a5e in term 1.
I20260808 02:17:49.766292 26095 leader_election.cc:302] T 4a42802275a84bb0962691d95fbfe7c5 P 1799c5cbe3ea4ef7821fb0e76a4d5a5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1799c5cbe3ea4ef7821fb0e76a4d5a5e, ec9bdcb6eb274a3d81481d180f0d92da; no voters:
I20260808 02:17:49.766484 26502 raft_consensus.cc:2781] T 4a42802275a84bb0962691d95fbfe7c5 P 1799c5cbe3ea4ef7821fb0e76a4d5a5e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:49.766747 26502 raft_consensus.cc:686] T 4a42802275a84bb0962691d95fbfe7c5 P 1799c5cbe3ea4ef7821fb0e76a4d5a5e [term 1 LEADER]: Becoming Leader. State: Replica: 1799c5cbe3ea4ef7821fb0e76a4d5a5e, State: Running, Role: LEADER
I20260808 02:17:49.767190 26502 consensus_queue.cc:237] T 4a42802275a84bb0962691d95fbfe7c5 P 1799c5cbe3ea4ef7821fb0e76a4d5a5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9bdcb6eb274a3d81481d180f0d92da" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 36693 } } peers { permanent_uuid: "1799c5cbe3ea4ef7821fb0e76a4d5a5e" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 38219 } } peers { permanent_uuid: "4ddda691326c441eb58c3f1045a2e67d" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 35943 } }
I20260808 02:17:49.771064 25782 catalog_manager.cc:5495] T 4a42802275a84bb0962691d95fbfe7c5 P 1799c5cbe3ea4ef7821fb0e76a4d5a5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 1799c5cbe3ea4ef7821fb0e76a4d5a5e (127.24.79.194). New cstate: current_term: 1 leader_uuid: "1799c5cbe3ea4ef7821fb0e76a4d5a5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9bdcb6eb274a3d81481d180f0d92da" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 36693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1799c5cbe3ea4ef7821fb0e76a4d5a5e" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 38219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ddda691326c441eb58c3f1045a2e67d" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 35943 } health_report { overall_health: UNKNOWN } } }
Aug 08 02:17:49 dist-test-slave-1vmg krb5kdc[25713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.193@KRBTEST.COM
I20260808 02:17:49.800144 26486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:49.785949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33026 (local address 127.24.79.193:35943)
0808 02:17:49.786097 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:49.786102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:49.786295 (+ 193us) server_negotiation.cc:407] Connection header received
0808 02:17:49.786438 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:49.786440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:49.786485 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:49.786554 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:49.787308 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.787827 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.788671 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.788900 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.788991 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:49.796218 (+ 7227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:49.796240 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:49.796266 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:49.796307 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:49.798069 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:49.798617 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:49.798623 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:49.798624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:49.798709 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:49.799016 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:49.799019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:49.799020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:49.799313 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:49.799485 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:49.799878 (+ 393us) server_negotiation.cc:299] Negotiation successful
0808 02:17:49.800008 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 02:17:49 dist-test-slave-1vmg krb5kdc[25713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155469, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:17:49 dist-test-slave-1vmg krb5kdc[25713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155469, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.79.193@KRBTEST.COM
I20260808 02:17:49.832544 26486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:49.822984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33032 (local address 127.24.79.193:35943)
0808 02:17:49.823154 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:49.823160 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:49.823355 (+ 195us) server_negotiation.cc:407] Connection header received
0808 02:17:49.823459 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:49.823462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:49.823514 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:49.823580 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:49.824368 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.824919 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.825935 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.826183 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.826281 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:49.829115 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:49.829120 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:49.829122 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:49.829159 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:49.830806 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:49.831397 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:49.831400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:49.831401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:49.831500 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:49.831725 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:49.831726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:49.831727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:49.831917 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:49.832033 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:49.832269 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 02:17:49.832386 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 02:17:49 dist-test-slave-1vmg krb5kdc[25713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 02:17:49 dist-test-slave-1vmg krb5kdc[25713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.193@KRBTEST.COM
I20260808 02:17:49.865849 26486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:49.856532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33046 (local address 127.24.79.193:35943)
0808 02:17:49.856674 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:49.856679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:49.856858 (+ 179us) server_negotiation.cc:407] Connection header received
0808 02:17:49.857021 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:49.857024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:49.857072 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:49.857139 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:49.857944 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.858458 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.859392 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:49.859618 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:49.859723 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:49.862530 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:49.862533 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:49.862535 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:49.862567 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:49.864109 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:49.864570 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:49.864574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:49.864575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:49.864664 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:49.864944 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:49.864947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:49.864949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:49.865146 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:49.865269 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:49.865540 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 02:17:49.865673 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 02:17:49.867863 25929 tablet_service.cc:2908] Rejecting scan request for tablet 4a42802275a84bb0962691d95fbfe7c5: Uninitialized: clean time has not yet been initialized
I20260808 02:17:49.870489 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 25866
I20260808 02:17:49.883484 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 26068
W20260808 02:17:49.895388 26465 tablet.cc:2369] T 4a42802275a84bb0962691d95fbfe7c5 P ec9bdcb6eb274a3d81481d180f0d92da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:17:49.897399 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 26269
I20260808 02:17:49.906966 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 25736
2026-08-08T02:17:49Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2542 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:49 dist-test-slave-1vmg krb5kdc[26526](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:49 dist-test-slave-1vmg krb5kdc[26526](info): set up 2 sockets
Aug 08 02:17:49 dist-test-slave-1vmg krb5kdc[26526](info): commencing operation
krb5kdc: starting...
W20260808 02:17:51.050856 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:17:51 dist-test-slave-1vmg krb5kdc[26526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:17:51.190948 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:42441
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:42441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:51.315783 26543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:51.316121 26543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:51.316172 26543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:51.320402 26543 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:51.320485 26543 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 02:17:51.320502 26543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:51.320520 26543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:51.320535 26543 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:51.320549 26543 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:51.326901 26543 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:42441
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:42441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:51.328616 26543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:51.330106 26543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:51.345191 26557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:51.345223 26550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:51.345222 26554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:51.345454 26543 server_base.cc:1034] running on GCE node
I20260808 02:17:51.346248 26543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:51.347582 26543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:51.348778 26543 hybrid_clock.cc:648] HybridClock initialized: now 1786155471348738 us; error 57 us; skew 500 ppm
Aug 08 02:17:51 dist-test-slave-1vmg krb5kdc[26526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155471, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:51.351857 26543 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:17:51.352875 26543 webserver.cc:460] Webserver started at http://127.24.79.254:40897/ using document root <none> and password file <none>
I20260808 02:17:51.353462 26543 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:51.353528 26543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:51.353708 26543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:51.356072 26543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "811b94b7fcdf4dfda3fe111460c0048b"
format_stamp: "Formatted at 2026-08-08 02:17:51 on dist-test-slave-1vmg"
server_key: "9c787782fa884da61c35a6d6e8a65155"
server_key_iv: "7e8f385b1c9eb05815731f919865aa96"
server_key_version: "encryptionkey@0"
I20260808 02:17:51.356667 26543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "811b94b7fcdf4dfda3fe111460c0048b"
format_stamp: "Formatted at 2026-08-08 02:17:51 on dist-test-slave-1vmg"
server_key: "9c787782fa884da61c35a6d6e8a65155"
server_key_iv: "7e8f385b1c9eb05815731f919865aa96"
server_key_version: "encryptionkey@0"
I20260808 02:17:51.361092 26543 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:17:51.363955 26569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:51.365410 26543 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:17:51.365545 26543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "811b94b7fcdf4dfda3fe111460c0048b"
format_stamp: "Formatted at 2026-08-08 02:17:51 on dist-test-slave-1vmg"
server_key: "9c787782fa884da61c35a6d6e8a65155"
server_key_iv: "7e8f385b1c9eb05815731f919865aa96"
server_key_version: "encryptionkey@0"
I20260808 02:17:51.365680 26543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:51.381490 26543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:51.385046 26543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:51.385243 26543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:51.394199 26543 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:42441
I20260808 02:17:51.394241 26662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:42441 every 8 connection(s)
I20260808 02:17:51.395325 26543 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:51.399732 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 26543
I20260808 02:17:51.399814 26663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:51.399878 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:17:51.400189 24895 external_mini_cluster.cc:1428] Setting key b6525da8d0a2678c361f8cfcc28c7b7f
I20260808 02:17:51.407577 26663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b: Bootstrap starting.
Aug 08 02:17:51 dist-test-slave-1vmg krb5kdc[26526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:17:51.410522 26663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:51.411376 26663 log.cc:824] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:51.413759 26663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b: No bootstrap required, opened a new log
I20260808 02:17:51.417210 26666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:51.401708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56714 (local address 127.24.79.254:42441)
0808 02:17:51.402197 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:51.402215 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:51.402257 (+ 42us) server_negotiation.cc:407] Connection header received
0808 02:17:51.403242 (+ 985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:51.403267 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:51.403678 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:51.403996 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:51.405029 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:51.406135 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:51.407436 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:51.408050 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:51.408203 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:51.411173 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:51.411179 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:51.411196 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:51.411232 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:51.413871 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:51.414521 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:51.414526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:51.414532 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:51.414687 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:51.415050 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:51.415052 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:51.415053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:51.415507 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:51.415678 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:51.416075 (+ 397us) server_negotiation.cc:299] Negotiation successful
0808 02:17:51.416415 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":149,"threads_started":1}
I20260808 02:17:51.417644 26663 raft_consensus.cc:348] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "811b94b7fcdf4dfda3fe111460c0048b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 42441 } }
I20260808 02:17:51.417907 26663 raft_consensus.cc:374] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:51.417997 26663 raft_consensus.cc:729] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 811b94b7fcdf4dfda3fe111460c0048b, State: Initialized, Role: FOLLOWER
I20260808 02:17:51.418524 26663 consensus_queue.cc:260] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "811b94b7fcdf4dfda3fe111460c0048b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 42441 } }
I20260808 02:17:51.418726 26663 raft_consensus.cc:388] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:51.418815 26663 raft_consensus.cc:482] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:51.418969 26663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:51.420228 26663 raft_consensus.cc:504] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "811b94b7fcdf4dfda3fe111460c0048b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 42441 } }
I20260808 02:17:51.420576 26663 leader_election.cc:302] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 811b94b7fcdf4dfda3fe111460c0048b; no voters:
I20260808 02:17:51.420936 26663 leader_election.cc:288] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:51.421042 26669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:51.421315 26669 raft_consensus.cc:686] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b [term 1 LEADER]: Becoming Leader. State: Replica: 811b94b7fcdf4dfda3fe111460c0048b, State: Running, Role: LEADER
I20260808 02:17:51.421667 26669 consensus_queue.cc:237] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "811b94b7fcdf4dfda3fe111460c0048b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 42441 } }
I20260808 02:17:51.422259 26663 sys_catalog.cc:564] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:51.423501 26673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "811b94b7fcdf4dfda3fe111460c0048b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "811b94b7fcdf4dfda3fe111460c0048b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 42441 } } }
I20260808 02:17:51.423676 26673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:51.424003 26676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 811b94b7fcdf4dfda3fe111460c0048b. Latest consensus state: current_term: 1 leader_uuid: "811b94b7fcdf4dfda3fe111460c0048b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "811b94b7fcdf4dfda3fe111460c0048b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 42441 } } }
I20260808 02:17:51.424150 26676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:51.424206 26683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:51.427763 26683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:51.434592 26683 catalog_manager.cc:1347] Generated new cluster ID: d124424116d84167b1f08a66f2caecd2
I20260808 02:17:51.434685 26683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:51.440670 26683 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:51.441745 26683 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:51.455355 26683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b: Generated new TSK 0
I20260808 02:17:51.456161 26683 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.79.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
I20260808 02:17:51.493597 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:42441
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:51.616159 26702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:51.616499 26702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:51.616568 26702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:51.620813 26702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:51.620899 26702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:51.620955 26702 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:17:51.621009 26702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:17:51.627277 26702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.79.254:42441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:51.628585 26702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:51.629786 26702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:51.640658 26708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:51.640658 26709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:51.640662 26711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:51.641160 26702 server_base.cc:1034] running on GCE node
I20260808 02:17:51.641681 26702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:51.642396 26702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:51.643594 26702 hybrid_clock.cc:648] HybridClock initialized: now 1786155471643545 us; error 46 us; skew 500 ppm
Aug 08 02:17:51 dist-test-slave-1vmg krb5kdc[26526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155471, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:51.646651 26702 init.cc:377] Logged in from keytab as kudu/127.24.79.193@KRBTEST.COM (short username kudu)
I20260808 02:17:51.647689 26702 webserver.cc:460] Webserver started at http://127.24.79.193:35841/ using document root <none> and password file <none>
I20260808 02:17:51.648277 26702 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:51.648350 26702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:51.648595 26702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:51.650866 26702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "c3dc2031d5af479ca10a5db2c3101eb5"
format_stamp: "Formatted at 2026-08-08 02:17:51 on dist-test-slave-1vmg"
server_key: "9e354d698736ffef680111095c1e389b"
server_key_iv: "16cd77f5b9fb17822c79c05b80bcb6b9"
server_key_version: "encryptionkey@0"
I20260808 02:17:51.651548 26702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "c3dc2031d5af479ca10a5db2c3101eb5"
format_stamp: "Formatted at 2026-08-08 02:17:51 on dist-test-slave-1vmg"
server_key: "9e354d698736ffef680111095c1e389b"
server_key_iv: "16cd77f5b9fb17822c79c05b80bcb6b9"
server_key_version: "encryptionkey@0"
I20260808 02:17:51.655553 26702 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:17:51.658214 26719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:51.659783 26702 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:17:51.659931 26702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "c3dc2031d5af479ca10a5db2c3101eb5"
format_stamp: "Formatted at 2026-08-08 02:17:51 on dist-test-slave-1vmg"
server_key: "9e354d698736ffef680111095c1e389b"
server_key_iv: "16cd77f5b9fb17822c79c05b80bcb6b9"
server_key_version: "encryptionkey@0"
I20260808 02:17:51.660053 26702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:51.672688 26702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:51.676380 26702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:51.676599 26702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:51.678330 26702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:17:51.678406 26702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:51.678467 26702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:17:51.678494 26702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:17:51 dist-test-slave-1vmg krb5kdc[26526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155471, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:17:51.690047 26702 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:41261
I20260808 02:17:51.690100 26881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:41261 every 8 connection(s)
I20260808 02:17:51.691155 26702 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:17:51.692229 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 26702
I20260808 02:17:51.692332 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:17:51.692625 24895 external_mini_cluster.cc:1428] Setting key b41f6743ad1cd5c5422b3b23763412b1
I20260808 02:17:51.693470 26666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:51.679389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:43711 (local address 127.24.79.254:42441)
0808 02:17:51.679522 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:51.679525 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:51.680798 (+ 1273us) server_negotiation.cc:407] Connection header received
0808 02:17:51.682027 (+ 1229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:51.682033 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:51.682111 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:51.682228 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:51.683745 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:51.684403 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:51.686035 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:51.686294 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:51.686392 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:51.689237 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:51.689240 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:51.689291 (+ 51us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:51.689324 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:51.690902 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:51.691530 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:51.691533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:51.691534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:51.691647 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:51.692076 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:51.692080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:51.692090 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:51.692377 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:51.692480 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:51.693187 (+ 707us) server_negotiation.cc:299] Negotiation successful
0808 02:17:51.693332 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 02:17:51.694442 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:51.680025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:42441 (local address 127.24.79.193:43711)
0808 02:17:51.680614 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:51.680665 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:51.681697 (+ 1032us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:51.682453 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:51.682463 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:51.682962 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:51.683545 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:51.683561 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:51.684547 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:51.684550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:51.685872 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:51.685885 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:51.686477 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:51.686481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:51.686655 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:51.687352 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:51.687371 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:51.689029 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:51.691058 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:51.691065 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:51.691083 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:51.691416 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:51.691763 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:51.691766 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:51.691768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:51.691949 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:51.692492 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:51.692499 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:51.692853 (+ 354us) client_negotiation.cc:769] Sending connection context
0808 02:17:51.693079 (+ 226us) client_negotiation.cc:240] Negotiation successful
0808 02:17:51.693439 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":153,"threads_started":1}
I20260808 02:17:51.696020 26882 heartbeater.cc:347] Connected to a master server at 127.24.79.254:42441
I20260808 02:17:51.696197 26882 heartbeater.cc:464] Registering TS with master...
I20260808 02:17:51.696882 26882 heartbeater.cc:511] Master 127.24.79.254:42441 requested a full tablet report, sending...
I20260808 02:17:51.699120 26593 ts_manager.cc:194] Registered new tserver with Master: c3dc2031d5af479ca10a5db2c3101eb5 (127.24.79.193:41261)
I20260808 02:17:51.700415 26593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.193@KRBTEST.COM'} at 127.24.79.193:43711
WARNING: no policy specified for kudu/127.24.79.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.194@KRBTEST.COM" created.
I20260808 02:17:51.710567 26666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:51.698792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:58009 (local address 127.24.79.254:42441)
0808 02:17:51.698954 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:51.698958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:51.703354 (+ 4396us) server_negotiation.cc:407] Connection header received
0808 02:17:51.703504 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:51.703507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:51.703554 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:51.703622 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:51.704156 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:51.704688 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:51.705630 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:51.705892 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:51.706010 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:51.707336 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:51.707339 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:51.707341 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:51.707367 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:51.708858 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:51.709304 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:51.709307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:51.709308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:51.709391 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:51.709662 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:51.709665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:51.709667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:51.709877 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:51.709970 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:51.710289 (+ 319us) server_negotiation.cc:299] Negotiation successful
0808 02:17:51.710415 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 02:17:51.710579 26895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:51.698708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:42441 (local address 127.24.79.193:58009)
0808 02:17:51.703218 (+ 4510us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:51.703237 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:51.703388 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:51.703667 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:51.703670 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:51.703907 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:51.704020 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:51.704028 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:51.704844 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:51.704849 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:51.705490 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:51.705502 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:51.706048 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:51.706051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:51.706138 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:51.706687 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:51.706699 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:51.707167 (+ 468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:51.708975 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:51.708978 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:51.708981 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:51.709195 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:51.709482 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:51.709485 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:51.709487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:51.709576 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:51.709977 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:51.709980 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:51.710083 (+ 103us) client_negotiation.cc:769] Sending connection context
0808 02:17:51.710173 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 02:17:51.710350 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4393,"thread_start_us":134,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
I20260808 02:17:51.727352 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:42441
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:51.851256 26898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:51.851550 26898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:51.851595 26898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:51.855625 26898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:51.855688 26898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:51.855726 26898 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:17:51.855760 26898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:17:51.861294 26898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.79.254:42441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:51.862483 26898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:51.863605 26898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:51.873922 26911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:51.873919 26914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:51.873919 26908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:51.874205 26898 server_base.cc:1034] running on GCE node
I20260808 02:17:51.874768 26898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:51.875552 26898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:51.876741 26898 hybrid_clock.cc:648] HybridClock initialized: now 1786155471876720 us; error 36 us; skew 500 ppm
Aug 08 02:17:51 dist-test-slave-1vmg krb5kdc[26526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155471, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:51.879812 26898 init.cc:377] Logged in from keytab as kudu/127.24.79.194@KRBTEST.COM (short username kudu)
I20260808 02:17:51.880784 26898 webserver.cc:460] Webserver started at http://127.24.79.194:45979/ using document root <none> and password file <none>
I20260808 02:17:51.881373 26898 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:51.881445 26898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:51.881664 26898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:51.883940 26898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "ea6afc0ee252416cb5f2a31717dc3ac6"
format_stamp: "Formatted at 2026-08-08 02:17:51 on dist-test-slave-1vmg"
server_key: "873c1ee0eb7944ca76269e9f580c1a6b"
server_key_iv: "bec31b308d4d530488d08de0982b3079"
server_key_version: "encryptionkey@0"
I20260808 02:17:51.884518 26898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "ea6afc0ee252416cb5f2a31717dc3ac6"
format_stamp: "Formatted at 2026-08-08 02:17:51 on dist-test-slave-1vmg"
server_key: "873c1ee0eb7944ca76269e9f580c1a6b"
server_key_iv: "bec31b308d4d530488d08de0982b3079"
server_key_version: "encryptionkey@0"
I20260808 02:17:51.888578 26898 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:17:51.891346 26923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:51.892701 26898 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:17:51.892839 26898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "ea6afc0ee252416cb5f2a31717dc3ac6"
format_stamp: "Formatted at 2026-08-08 02:17:51 on dist-test-slave-1vmg"
server_key: "873c1ee0eb7944ca76269e9f580c1a6b"
server_key_iv: "bec31b308d4d530488d08de0982b3079"
server_key_version: "encryptionkey@0"
I20260808 02:17:51.892961 26898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:51.914551 26898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:51.918282 26898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:51.918506 26898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:51.920286 26898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:17:51.920351 26898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:51.920413 26898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:17:51.920442 26898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:51.931550 26898 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:40929
I20260808 02:17:51.931644 27095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:40929 every 8 connection(s)
Aug 08 02:17:51 dist-test-slave-1vmg krb5kdc[26526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155471, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:17:51.932672 26898 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:17:51.936568 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 26898
I20260808 02:17:51.936671 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:17:51.936856 26666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:51.921273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:33499 (local address 127.24.79.254:42441)
0808 02:17:51.921404 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:51.921407 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:51.922577 (+ 1170us) server_negotiation.cc:407] Connection header received
0808 02:17:51.923894 (+ 1317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:51.923899 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:51.923974 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:51.924089 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:51.925826 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:51.926650 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:51.928523 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:51.928896 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:51.929031 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:51.932463 (+ 3432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:51.932467 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:51.932470 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:51.932505 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:51.934359 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:51.935020 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:51.935024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:51.935025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:51.935135 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:51.935544 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:51.935547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:51.935548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:51.935769 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:51.935873 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:51.936502 (+ 629us) server_negotiation.cc:299] Negotiation successful
0808 02:17:51.936666 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 02:17:51.936919 24895 external_mini_cluster.cc:1428] Setting key ad1634cac1536ee05c0cb4b572263041
I20260808 02:17:51.937942 26937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:51.921861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:42441 (local address 127.24.79.194:33499)
0808 02:17:51.922417 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:51.922468 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:51.923556 (+ 1088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:51.924311 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:51.924323 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:51.924780 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:51.925591 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:51.925617 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:51.926822 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:51.926826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:51.928369 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:51.928382 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:51.929191 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:51.929203 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:51.929454 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:51.930263 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:51.930290 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:51.932216 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:51.934501 (+ 2285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:51.934509 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:51.934535 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:51.934846 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:51.935259 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:51.935262 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:51.935263 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:51.935425 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:51.935898 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:51.935906 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:51.936197 (+ 291us) client_negotiation.cc:769] Sending connection context
0808 02:17:51.936415 (+ 218us) client_negotiation.cc:240] Negotiation successful
0808 02:17:51.936823 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":135,"threads_started":1}
I20260808 02:17:51.941468 27096 heartbeater.cc:347] Connected to a master server at 127.24.79.254:42441
I20260808 02:17:51.941653 27096 heartbeater.cc:464] Registering TS with master...
I20260808 02:17:51.942467 27096 heartbeater.cc:511] Master 127.24.79.254:42441 requested a full tablet report, sending...
I20260808 02:17:51.943668 26595 ts_manager.cc:194] Registered new tserver with Master: ea6afc0ee252416cb5f2a31717dc3ac6 (127.24.79.194:40929)
I20260808 02:17:51.944376 26595 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.194@KRBTEST.COM'} at 127.24.79.194:33499
WARNING: no policy specified for kudu/127.24.79.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.195@KRBTEST.COM" created.
I20260808 02:17:51.951985 27107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:51.941794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:42441 (local address 127.24.79.194:51797)
0808 02:17:51.942163 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:51.942180 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:51.942349 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:51.942733 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:51.942736 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:51.942996 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:51.943127 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:51.943135 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:51.945386 (+ 2251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:51.945391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:51.946328 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:51.946341 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:51.946996 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:51.947000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:51.947103 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:51.947706 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:51.947721 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:51.948186 (+ 465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:51.950098 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:51.950101 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:51.950103 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:51.950362 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:51.950682 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:51.950685 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:51.950687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:51.950783 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:51.951449 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:51.951452 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:51.951558 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 02:17:51.951654 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 02:17:51.951807 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":1,"thread_start_us":150,"threads_started":1}
I20260808 02:17:51.952162 26666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:51.941798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:51797 (local address 127.24.79.254:42441)
0808 02:17:51.941922 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:51.941927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:51.942271 (+ 344us) server_negotiation.cc:407] Connection header received
0808 02:17:51.942467 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:51.942471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:51.942526 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:51.942605 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:51.943273 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:51.945224 (+ 1951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:51.946489 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:51.946812 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:51.946971 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:51.948391 (+ 1420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:51.948395 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:51.948397 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:51.948424 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:51.949966 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:51.950471 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:51.950474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:51.950475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:51.950565 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:51.950887 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:51.950891 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:51.950893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:51.951278 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:51.951429 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:51.951762 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 02:17:51.951909 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
I20260808 02:17:51.970824 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:42441
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:52.096439 27109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:52.096784 27109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:52.096834 27109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:52.101153 27109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:52.101257 27109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:52.101302 27109 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:17:52.101338 27109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:17:52.107162 27109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.79.254:42441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:52.108387 27109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:52.109575 27109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:52.120391 27117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:52.120391 27116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:52.120431 27109 server_base.cc:1034] running on GCE node
W20260808 02:17:52.120393 27119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:52.121227 27109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:52.122005 27109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:52.123191 27109 hybrid_clock.cc:648] HybridClock initialized: now 1786155472123163 us; error 46 us; skew 500 ppm
Aug 08 02:17:52 dist-test-slave-1vmg krb5kdc[26526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155472, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:52.126417 27109 init.cc:377] Logged in from keytab as kudu/127.24.79.195@KRBTEST.COM (short username kudu)
I20260808 02:17:52.127560 27109 webserver.cc:460] Webserver started at http://127.24.79.195:41305/ using document root <none> and password file <none>
I20260808 02:17:52.128185 27109 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:52.128301 27109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:52.128542 27109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:52.130965 27109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "d9539d61446245fc8c8cc82470b7298d"
format_stamp: "Formatted at 2026-08-08 02:17:52 on dist-test-slave-1vmg"
server_key: "24f4da3ed8a2e8ea872b5dfd2dadf5d1"
server_key_iv: "f67f1a042573ea695470a7241d60d1dc"
server_key_version: "encryptionkey@0"
I20260808 02:17:52.131698 27109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "d9539d61446245fc8c8cc82470b7298d"
format_stamp: "Formatted at 2026-08-08 02:17:52 on dist-test-slave-1vmg"
server_key: "24f4da3ed8a2e8ea872b5dfd2dadf5d1"
server_key_iv: "f67f1a042573ea695470a7241d60d1dc"
server_key_version: "encryptionkey@0"
I20260808 02:17:52.136272 27109 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:17:52.139468 27131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:52.140983 27109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:17:52.141139 27109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "d9539d61446245fc8c8cc82470b7298d"
format_stamp: "Formatted at 2026-08-08 02:17:52 on dist-test-slave-1vmg"
server_key: "24f4da3ed8a2e8ea872b5dfd2dadf5d1"
server_key_iv: "f67f1a042573ea695470a7241d60d1dc"
server_key_version: "encryptionkey@0"
I20260808 02:17:52.141304 27109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:52.153442 27109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:52.157055 27109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:52.157265 27109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:52.159049 27109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:17:52.159109 27109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:52.159168 27109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:17:52.159205 27109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:17:52 dist-test-slave-1vmg krb5kdc[26526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155472, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:17:52.170631 27109 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:37987
I20260808 02:17:52.170675 27306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:37987 every 8 connection(s)
I20260808 02:17:52.171907 27109 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:17:52.174350 26666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.159932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:39325 (local address 127.24.79.254:42441)
0808 02:17:52.160067 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:52.160072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:52.161275 (+ 1203us) server_negotiation.cc:407] Connection header received
0808 02:17:52.162353 (+ 1078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:52.162357 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:52.162422 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:52.162483 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:52.164251 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.165101 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.166441 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.166802 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.166948 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.170115 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:52.170119 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:52.170122 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:52.170158 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:52.172025 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:52.172647 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:52.172650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:52.172652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:52.172760 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:52.173183 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:52.173186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:52.173187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:52.173403 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:52.173493 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:52.174085 (+ 592us) server_negotiation.cc:299] Negotiation successful
0808 02:17:52.174216 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 02:17:52.175426 27146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.160496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:42441 (local address 127.24.79.195:39325)
0808 02:17:52.161095 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:52.161144 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:52.162060 (+ 916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:52.162875 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:52.162882 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:52.163311 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:52.164053 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.164088 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.165252 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.165255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.166318 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.166330 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.167092 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.167097 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.167315 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.168013 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:52.168032 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:52.169886 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:52.172152 (+ 2266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:52.172159 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:52.172176 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:52.172510 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:52.172851 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:52.172854 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:52.172856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:52.173059 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:52.173526 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:52.173533 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:52.173839 (+ 306us) client_negotiation.cc:769] Sending connection context
0808 02:17:52.174018 (+ 179us) client_negotiation.cc:240] Negotiation successful
0808 02:17:52.174338 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":168,"threads_started":1}
I20260808 02:17:52.177116 27307 heartbeater.cc:347] Connected to a master server at 127.24.79.254:42441
I20260808 02:17:52.177330 27307 heartbeater.cc:464] Registering TS with master...
I20260808 02:17:52.177983 27307 heartbeater.cc:511] Master 127.24.79.254:42441 requested a full tablet report, sending...
I20260808 02:17:52.179256 26595 ts_manager.cc:194] Registered new tserver with Master: d9539d61446245fc8c8cc82470b7298d (127.24.79.195:37987)
I20260808 02:17:52.180023 26595 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.195@KRBTEST.COM'} at 127.24.79.195:39325
I20260808 02:17:52.180598 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 27109
I20260808 02:17:52.180737 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:17:52.181051 24895 external_mini_cluster.cc:1428] Setting key 0edef014f288c2c0ad0177d70787dffb
I20260808 02:17:52.183302 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:17:52.187708 26666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.179480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:56685 (local address 127.24.79.254:42441)
0808 02:17:52.179612 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:52.179617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:52.179784 (+ 167us) server_negotiation.cc:407] Connection header received
0808 02:17:52.179910 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:52.179913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:52.179960 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:52.180040 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:52.180730 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.181486 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.182565 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.182875 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.183025 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.184392 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:52.184396 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:52.184400 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:52.184430 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:52.186000 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:52.186432 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:52.186436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:52.186438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:52.186524 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:52.186810 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:52.186813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:52.186815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:52.187057 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:52.187149 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:52.187419 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 02:17:52.187533 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 02:17:52.187733 27313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.179395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:42441 (local address 127.24.79.195:56685)
0808 02:17:52.179698 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:52.179714 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:52.179820 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:52.180154 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:52.180158 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:52.180410 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:52.180563 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.180572 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.181657 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.181660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.182447 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.182458 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.183030 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.183034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.183190 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.183707 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:52.183721 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:52.184191 (+ 470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:52.186117 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:52.186121 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:52.186123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:52.186335 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:52.186609 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:52.186613 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:52.186615 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:52.186721 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:52.187149 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:52.187152 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:52.187256 (+ 104us) client_negotiation.cc:769] Sending connection context
0808 02:17:52.187349 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 02:17:52.187523 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":125,"threads_started":1}
Aug 08 02:17:52 dist-test-slave-1vmg krb5kdc[26526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155472, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:17:52 dist-test-slave-1vmg krb5kdc[26526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155472, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:17:52.213222 26666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.204320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56722 (local address 127.24.79.254:42441)
0808 02:17:52.204496 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:52.204501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:52.204516 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:17:52.204567 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:52.204569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:52.204611 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:52.204695 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:52.205496 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.206044 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.206999 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.207254 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.207350 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.209996 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:52.210000 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:52.210002 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:52.210034 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:52.211412 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:52.211806 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:52.211808 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:52.211809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:52.211886 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:52.212134 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:52.212138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:52.212139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:52.212417 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:52.212552 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:52.212894 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 02:17:52.213033 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 02:17:52.216574 26595 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56722:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:17:52.219718 26595 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:17:52.236212 27333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.229448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:37987 (local address 127.0.0.1:39836)
0808 02:17:52.229917 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:52.229940 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:52.230093 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:52.231115 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:52.231121 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:52.231148 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:52.231347 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.231356 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.233088 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.233091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.234627 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.234638 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.235590 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.235594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.235715 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.235768 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 02:17:52.235888 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 02:17:52.236032 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":151,"spinlock_wait_cycles":3328,"thread_start_us":149,"threads_started":1}
I20260808 02:17:52.236402 27336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.229460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60538 (local address 127.24.79.194:40929)
0808 02:17:52.230856 (+ 1396us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:52.230867 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:52.230891 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:17:52.230992 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:52.230997 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:52.231245 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:52.231702 (+ 457us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:52.231840 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.232930 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.234821 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.235811 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.235918 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.236074 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:52.236135 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 02:17:52.236248 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1295,"thread_start_us":152,"threads_started":1}
I20260808 02:17:52.236567 27340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.230200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39836 (local address 127.24.79.195:37987)
0808 02:17:52.230544 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:52.230556 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:52.230582 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:17:52.230664 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:52.230669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:52.230902 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:52.231506 (+ 604us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:52.231662 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.232924 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.234752 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.235475 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.235576 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.235784 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:52.236304 (+ 520us) server_negotiation.cc:299] Negotiation successful
0808 02:17:52.236416 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":124,"threads_started":1}
I20260808 02:17:52.237002 27338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.230108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:41261 (local address 127.0.0.1:47110)
0808 02:17:52.230354 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:52.230368 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:52.230453 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:52.231575 (+ 1122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:52.231577 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:52.231592 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:52.231710 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.231715 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.233041 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.233045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.235325 (+ 2280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.235336 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.236592 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.236596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.236689 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.236720 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 02:17:52.236797 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 02:17:52.236872 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":85,"thread_start_us":117,"threads_started":1}
I20260808 02:17:52.237052 27335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.229280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:40929 (local address 127.0.0.1:60538)
0808 02:17:52.229880 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:52.229938 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:52.230093 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:52.231494 (+ 1401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:52.231497 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:52.231517 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:52.231690 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.231699 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.233087 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.233091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.234673 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.234685 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.235918 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.235921 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.236021 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.236042 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 02:17:52.236103 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 02:17:52.236158 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":154,"spinlock_wait_cycles":10752,"thread_start_us":126,"threads_started":1}
I20260808 02:17:52.237866 27343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.230570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47110 (local address 127.24.79.193:41261)
0808 02:17:52.230988 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:52.230995 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:52.231019 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:17:52.231093 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:52.231098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:52.231299 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:52.231446 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:52.231837 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.232902 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.235478 (+ 2576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.236280 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.236382 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.236522 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:52.237624 (+ 1102us) server_negotiation.cc:299] Negotiation successful
0808 02:17:52.237735 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":80,"threads_started":1}
I20260808 02:17:52.240039 27206 tablet_service.cc:1467] Processing CreateTablet for tablet 82d376e7451b440bb5b159d2b20266e1 (DEFAULT_TABLE table=test-table [id=0c08f90bf32045498e52736f20bcf758]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:52.240087 26996 tablet_service.cc:1467] Processing CreateTablet for tablet 82d376e7451b440bb5b159d2b20266e1 (DEFAULT_TABLE table=test-table [id=0c08f90bf32045498e52736f20bcf758]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:52.240566 26793 tablet_service.cc:1467] Processing CreateTablet for tablet 82d376e7451b440bb5b159d2b20266e1 (DEFAULT_TABLE table=test-table [id=0c08f90bf32045498e52736f20bcf758]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:52.241179 27206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82d376e7451b440bb5b159d2b20266e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:52.241180 26996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82d376e7451b440bb5b159d2b20266e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:52.241582 26793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82d376e7451b440bb5b159d2b20266e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:52.249655 27345 tablet_bootstrap.cc:492] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5: Bootstrap starting.
I20260808 02:17:52.250687 27348 tablet_bootstrap.cc:492] T 82d376e7451b440bb5b159d2b20266e1 P d9539d61446245fc8c8cc82470b7298d: Bootstrap starting.
I20260808 02:17:52.251319 27346 tablet_bootstrap.cc:492] T 82d376e7451b440bb5b159d2b20266e1 P ea6afc0ee252416cb5f2a31717dc3ac6: Bootstrap starting.
I20260808 02:17:52.252118 27345 tablet_bootstrap.cc:654] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:52.252883 27345 log.cc:824] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:52.252925 27348 tablet_bootstrap.cc:654] T 82d376e7451b440bb5b159d2b20266e1 P d9539d61446245fc8c8cc82470b7298d: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:52.253505 27346 tablet_bootstrap.cc:654] T 82d376e7451b440bb5b159d2b20266e1 P ea6afc0ee252416cb5f2a31717dc3ac6: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:52.253765 27348 log.cc:824] T 82d376e7451b440bb5b159d2b20266e1 P d9539d61446245fc8c8cc82470b7298d: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:52.254326 27346 log.cc:824] T 82d376e7451b440bb5b159d2b20266e1 P ea6afc0ee252416cb5f2a31717dc3ac6: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:52.254885 27345 tablet_bootstrap.cc:492] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5: No bootstrap required, opened a new log
I20260808 02:17:52.255154 27345 ts_tablet_manager.cc:1397] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 02:17:52.256894 27346 tablet_bootstrap.cc:492] T 82d376e7451b440bb5b159d2b20266e1 P ea6afc0ee252416cb5f2a31717dc3ac6: No bootstrap required, opened a new log
I20260808 02:17:52.257105 27346 ts_tablet_manager.cc:1397] T 82d376e7451b440bb5b159d2b20266e1 P ea6afc0ee252416cb5f2a31717dc3ac6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 02:17:52.260011 27345 raft_consensus.cc:348] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } } peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } }
I20260808 02:17:52.260356 27345 raft_consensus.cc:374] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:52.260457 27345 raft_consensus.cc:729] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3dc2031d5af479ca10a5db2c3101eb5, State: Initialized, Role: FOLLOWER
I20260808 02:17:52.261008 27348 tablet_bootstrap.cc:492] T 82d376e7451b440bb5b159d2b20266e1 P d9539d61446245fc8c8cc82470b7298d: No bootstrap required, opened a new log
I20260808 02:17:52.261044 27345 consensus_queue.cc:260] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } } peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } }
I20260808 02:17:52.261276 27348 ts_tablet_manager.cc:1397] T 82d376e7451b440bb5b159d2b20266e1 P d9539d61446245fc8c8cc82470b7298d: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260808 02:17:52.261183 27346 raft_consensus.cc:348] T 82d376e7451b440bb5b159d2b20266e1 P ea6afc0ee252416cb5f2a31717dc3ac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } } peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } }
I20260808 02:17:52.261456 27346 raft_consensus.cc:374] T 82d376e7451b440bb5b159d2b20266e1 P ea6afc0ee252416cb5f2a31717dc3ac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:52.261519 27346 raft_consensus.cc:729] T 82d376e7451b440bb5b159d2b20266e1 P ea6afc0ee252416cb5f2a31717dc3ac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea6afc0ee252416cb5f2a31717dc3ac6, State: Initialized, Role: FOLLOWER
I20260808 02:17:52.262148 27346 consensus_queue.cc:260] T 82d376e7451b440bb5b159d2b20266e1 P ea6afc0ee252416cb5f2a31717dc3ac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } } peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } }
I20260808 02:17:52.262292 27345 ts_tablet_manager.cc:1428] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 02:17:52.262698 26882 heartbeater.cc:503] Master 127.24.79.254:42441 was elected leader, sending a full tablet report...
I20260808 02:17:52.263309 27346 ts_tablet_manager.cc:1428] T 82d376e7451b440bb5b159d2b20266e1 P ea6afc0ee252416cb5f2a31717dc3ac6: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 02:17:52.263722 27096 heartbeater.cc:503] Master 127.24.79.254:42441 was elected leader, sending a full tablet report...
I20260808 02:17:52.265683 27348 raft_consensus.cc:348] T 82d376e7451b440bb5b159d2b20266e1 P d9539d61446245fc8c8cc82470b7298d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } } peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } }
I20260808 02:17:52.265949 27348 raft_consensus.cc:374] T 82d376e7451b440bb5b159d2b20266e1 P d9539d61446245fc8c8cc82470b7298d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:52.266022 27348 raft_consensus.cc:729] T 82d376e7451b440bb5b159d2b20266e1 P d9539d61446245fc8c8cc82470b7298d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9539d61446245fc8c8cc82470b7298d, State: Initialized, Role: FOLLOWER
I20260808 02:17:52.266584 27348 consensus_queue.cc:260] T 82d376e7451b440bb5b159d2b20266e1 P d9539d61446245fc8c8cc82470b7298d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } } peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } }
I20260808 02:17:52.267805 27307 heartbeater.cc:503] Master 127.24.79.254:42441 was elected leader, sending a full tablet report...
I20260808 02:17:52.269101 27348 ts_tablet_manager.cc:1428] T 82d376e7451b440bb5b159d2b20266e1 P d9539d61446245fc8c8cc82470b7298d: Time spent starting tablet: real 0.008s user 0.001s sys 0.007s
I20260808 02:17:52.289762 27353 raft_consensus.cc:482] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:17:52.290004 27353 raft_consensus.cc:504] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } } peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } }
I20260808 02:17:52.291326 27353 leader_election.cc:288] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea6afc0ee252416cb5f2a31717dc3ac6 (127.24.79.194:40929), d9539d61446245fc8c8cc82470b7298d (127.24.79.195:37987)
I20260808 02:17:52.295972 27356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.291657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:37987 (local address 127.24.79.193:43549)
0808 02:17:52.292132 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:52.292146 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:52.292292 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:52.292536 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:52.292539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:52.292564 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:52.292733 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.292739 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.293573 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.293577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.294662 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.294674 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.295519 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.295523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.295622 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.295648 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 02:17:52.295717 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 02:17:52.295781 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":106,"spinlock_wait_cycles":14080,"thread_start_us":115,"threads_started":1}
I20260808 02:17:52.295990 27340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.291831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:43549 (local address 127.24.79.195:37987)
0808 02:17:52.292082 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:52.292087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:52.292236 (+ 149us) server_negotiation.cc:407] Connection header received
0808 02:17:52.292400 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:52.292403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:52.292445 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:52.292506 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:52.292852 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.293463 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.294783 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.295385 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.295502 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.295595 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:52.295765 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 02:17:52.295859 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260808 02:17:52.296070 27336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.291870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:55691 (local address 127.24.79.194:40929)
0808 02:17:52.292005 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:52.292010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:52.292217 (+ 207us) server_negotiation.cc:407] Connection header received
0808 02:17:52.292395 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:52.292398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:52.292441 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:52.292504 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:52.292835 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.293452 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.294844 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.295423 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.295564 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.295620 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:52.295861 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 02:17:52.295938 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 02:17:52.296569 27359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.291657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:40929 (local address 127.24.79.193:55691)
0808 02:17:52.292105 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:52.292120 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:52.292289 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:52.292532 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:52.292534 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:52.292562 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:52.292709 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.292716 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.293561 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.293565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.294682 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.294693 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.295596 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.295600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.295727 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.295755 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 02:17:52.295825 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 02:17:52.295893 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":84,"spinlock_wait_cycles":53888,"thread_start_us":142,"threads_started":1}
I20260808 02:17:52.296736 27237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82d376e7451b440bb5b159d2b20266e1" candidate_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9539d61446245fc8c8cc82470b7298d" is_pre_election: true
I20260808 02:17:52.297097 27237 raft_consensus.cc:2454] T 82d376e7451b440bb5b159d2b20266e1 P d9539d61446245fc8c8cc82470b7298d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3dc2031d5af479ca10a5db2c3101eb5 in term 0.
I20260808 02:17:52.297211 27028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82d376e7451b440bb5b159d2b20266e1" candidate_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" is_pre_election: true
I20260808 02:17:52.297497 27028 raft_consensus.cc:2454] T 82d376e7451b440bb5b159d2b20266e1 P ea6afc0ee252416cb5f2a31717dc3ac6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3dc2031d5af479ca10a5db2c3101eb5 in term 0.
I20260808 02:17:52.297616 26726 leader_election.cc:302] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3dc2031d5af479ca10a5db2c3101eb5, d9539d61446245fc8c8cc82470b7298d; no voters:
I20260808 02:17:52.297919 27353 raft_consensus.cc:2781] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:17:52.298018 27353 raft_consensus.cc:482] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:17:52.298080 27353 raft_consensus.cc:3037] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:52.299255 27353 raft_consensus.cc:504] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } } peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } }
I20260808 02:17:52.299610 27353 leader_election.cc:288] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5 [CANDIDATE]: Term 1 election: Requested vote from peers ea6afc0ee252416cb5f2a31717dc3ac6 (127.24.79.194:40929), d9539d61446245fc8c8cc82470b7298d (127.24.79.195:37987)
I20260808 02:17:52.300050 27237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82d376e7451b440bb5b159d2b20266e1" candidate_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9539d61446245fc8c8cc82470b7298d"
I20260808 02:17:52.300050 27028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82d376e7451b440bb5b159d2b20266e1" candidate_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6"
I20260808 02:17:52.300196 27237 raft_consensus.cc:3037] T 82d376e7451b440bb5b159d2b20266e1 P d9539d61446245fc8c8cc82470b7298d [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:52.300196 27028 raft_consensus.cc:3037] T 82d376e7451b440bb5b159d2b20266e1 P ea6afc0ee252416cb5f2a31717dc3ac6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:52.301684 27237 raft_consensus.cc:2454] T 82d376e7451b440bb5b159d2b20266e1 P d9539d61446245fc8c8cc82470b7298d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3dc2031d5af479ca10a5db2c3101eb5 in term 1.
I20260808 02:17:52.302112 27028 raft_consensus.cc:2454] T 82d376e7451b440bb5b159d2b20266e1 P ea6afc0ee252416cb5f2a31717dc3ac6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3dc2031d5af479ca10a5db2c3101eb5 in term 1.
I20260808 02:17:52.302104 26726 leader_election.cc:302] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3dc2031d5af479ca10a5db2c3101eb5, d9539d61446245fc8c8cc82470b7298d; no voters:
I20260808 02:17:52.302330 27353 raft_consensus.cc:2781] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:52.302639 27353 raft_consensus.cc:686] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5 [term 1 LEADER]: Becoming Leader. State: Replica: c3dc2031d5af479ca10a5db2c3101eb5, State: Running, Role: LEADER
I20260808 02:17:52.303089 27353 consensus_queue.cc:237] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } } peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } }
I20260808 02:17:52.306748 26593 catalog_manager.cc:5495] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3dc2031d5af479ca10a5db2c3101eb5 (127.24.79.193). New cstate: current_term: 1 leader_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } health_report { overall_health: HEALTHY } } }
I20260808 02:17:52.329249 27343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.325275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47120 (local address 127.24.79.193:41261)
0808 02:17:52.325420 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:52.325425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:52.325472 (+ 47us) server_negotiation.cc:407] Connection header received
0808 02:17:52.325722 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:52.325745 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:52.325808 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:52.325899 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:52.326173 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.326680 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.327648 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.327871 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.327964 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.328345 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:52.328827 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:52.328937 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:52.329029 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 02:17:52.329098 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:17:52.336309 27028 raft_consensus.cc:1261] T 82d376e7451b440bb5b159d2b20266e1 P ea6afc0ee252416cb5f2a31717dc3ac6 [term 1 FOLLOWER]: Refusing update from remote peer c3dc2031d5af479ca10a5db2c3101eb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:52.336694 27237 raft_consensus.cc:1261] T 82d376e7451b440bb5b159d2b20266e1 P d9539d61446245fc8c8cc82470b7298d [term 1 FOLLOWER]: Refusing update from remote peer c3dc2031d5af479ca10a5db2c3101eb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:52.337270 27360 consensus_queue.cc:1035] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:52.337450 27353 consensus_queue.cc:1035] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:52.347002 27364 mvcc.cc:205] Tried to move back new op lower bound from 7316092814680862720 to 7316092814556569600. Current Snapshot: MvccSnapshot[applied={T|T < 7316092814680862720}]
I20260808 02:17:52.361372 26593 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:17:52.361596 26593 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:17:52.364573 26593 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b: Sending DeleteTablet for 3 replicas of tablet 82d376e7451b440bb5b159d2b20266e1
I20260808 02:17:52.365551 26996 tablet_service.cc:1514] Processing DeleteTablet for tablet 82d376e7451b440bb5b159d2b20266e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:52 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:60538
I20260808 02:17:52.365759 26793 tablet_service.cc:1514] Processing DeleteTablet for tablet 82d376e7451b440bb5b159d2b20266e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:52 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:47110
I20260808 02:17:52.365793 27206 tablet_service.cc:1514] Processing DeleteTablet for tablet 82d376e7451b440bb5b159d2b20266e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:52 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:39836
I20260808 02:17:52.366333 27374 tablet_replica.cc:330] T 82d376e7451b440bb5b159d2b20266e1 P ea6afc0ee252416cb5f2a31717dc3ac6: stopping tablet replica
I20260808 02:17:52.366531 27375 tablet_replica.cc:330] T 82d376e7451b440bb5b159d2b20266e1 P d9539d61446245fc8c8cc82470b7298d: stopping tablet replica
I20260808 02:17:52.366840 27374 raft_consensus.cc:2229] T 82d376e7451b440bb5b159d2b20266e1 P ea6afc0ee252416cb5f2a31717dc3ac6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:17:52.366839 27375 raft_consensus.cc:2229] T 82d376e7451b440bb5b159d2b20266e1 P d9539d61446245fc8c8cc82470b7298d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:17:52.367280 27375 raft_consensus.cc:2258] T 82d376e7451b440bb5b159d2b20266e1 P d9539d61446245fc8c8cc82470b7298d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:52.367280 27374 raft_consensus.cc:2258] T 82d376e7451b440bb5b159d2b20266e1 P ea6afc0ee252416cb5f2a31717dc3ac6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:52.367422 26593 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56722:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 02:17:52.367712 27376 tablet_replica.cc:330] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5: stopping tablet replica
W20260808 02:17:52.367919 26593 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:17:52.368037 27376 raft_consensus.cc:2229] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:17:52.368443 27376 raft_consensus.cc:2258] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:52.369891 27376 ts_tablet_manager.cc:1905] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:17:52.370859 27374 ts_tablet_manager.cc:1905] T 82d376e7451b440bb5b159d2b20266e1 P ea6afc0ee252416cb5f2a31717dc3ac6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:17:52.373081 27376 ts_tablet_manager.cc:1918] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:17:52.373208 27376 log.cc:1196] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-0/wal/wals/82d376e7451b440bb5b159d2b20266e1
I20260808 02:17:52.373549 27376 ts_tablet_manager.cc:1939] T 82d376e7451b440bb5b159d2b20266e1 P c3dc2031d5af479ca10a5db2c3101eb5: Deleting consensus metadata
I20260808 02:17:52.373769 26996 tablet_service.cc:1467] Processing CreateTablet for tablet 7ff26678dfcf420a832e9ff714e507ee (DEFAULT_TABLE table=test-table [id=8edc4082e2f646d1837ee985c4ee95a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:52.374096 26996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ff26678dfcf420a832e9ff714e507ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:52.374759 27374 ts_tablet_manager.cc:1918] T 82d376e7451b440bb5b159d2b20266e1 P ea6afc0ee252416cb5f2a31717dc3ac6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:17:52.375021 27374 log.cc:1196] T 82d376e7451b440bb5b159d2b20266e1 P ea6afc0ee252416cb5f2a31717dc3ac6: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/82d376e7451b440bb5b159d2b20266e1
I20260808 02:17:52.375444 27374 ts_tablet_manager.cc:1939] T 82d376e7451b440bb5b159d2b20266e1 P ea6afc0ee252416cb5f2a31717dc3ac6: Deleting consensus metadata
I20260808 02:17:52.375514 26574 catalog_manager.cc:4841] TS c3dc2031d5af479ca10a5db2c3101eb5 (127.24.79.193:41261): tablet 82d376e7451b440bb5b159d2b20266e1 (table test-table [id=0c08f90bf32045498e52736f20bcf758]) successfully deleted
I20260808 02:17:52.376962 26570 catalog_manager.cc:4841] TS ea6afc0ee252416cb5f2a31717dc3ac6 (127.24.79.194:40929): tablet 82d376e7451b440bb5b159d2b20266e1 (table test-table [id=0c08f90bf32045498e52736f20bcf758]) successfully deleted
I20260808 02:17:52.377454 27375 ts_tablet_manager.cc:1905] T 82d376e7451b440bb5b159d2b20266e1 P d9539d61446245fc8c8cc82470b7298d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:17:52.377899 26793 tablet_service.cc:1467] Processing CreateTablet for tablet 7ff26678dfcf420a832e9ff714e507ee (DEFAULT_TABLE table=test-table [id=8edc4082e2f646d1837ee985c4ee95a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:52.377960 27206 tablet_service.cc:1467] Processing CreateTablet for tablet 7ff26678dfcf420a832e9ff714e507ee (DEFAULT_TABLE table=test-table [id=8edc4082e2f646d1837ee985c4ee95a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:52.378185 26793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ff26678dfcf420a832e9ff714e507ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:52.378377 27206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ff26678dfcf420a832e9ff714e507ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:52.380357 27375 ts_tablet_manager.cc:1918] T 82d376e7451b440bb5b159d2b20266e1 P d9539d61446245fc8c8cc82470b7298d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:17:52.380473 27375 log.cc:1196] T 82d376e7451b440bb5b159d2b20266e1 P d9539d61446245fc8c8cc82470b7298d: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/82d376e7451b440bb5b159d2b20266e1
I20260808 02:17:52.380806 27375 ts_tablet_manager.cc:1939] T 82d376e7451b440bb5b159d2b20266e1 P d9539d61446245fc8c8cc82470b7298d: Deleting consensus metadata
I20260808 02:17:52.381199 27346 tablet_bootstrap.cc:492] T 7ff26678dfcf420a832e9ff714e507ee P ea6afc0ee252416cb5f2a31717dc3ac6: Bootstrap starting.
I20260808 02:17:52.381620 27345 tablet_bootstrap.cc:492] T 7ff26678dfcf420a832e9ff714e507ee P c3dc2031d5af479ca10a5db2c3101eb5: Bootstrap starting.
I20260808 02:17:52.381943 27348 tablet_bootstrap.cc:492] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d: Bootstrap starting.
I20260808 02:17:52.382174 26573 catalog_manager.cc:4841] TS d9539d61446245fc8c8cc82470b7298d (127.24.79.195:37987): tablet 82d376e7451b440bb5b159d2b20266e1 (table test-table [id=0c08f90bf32045498e52736f20bcf758]) successfully deleted
I20260808 02:17:52.382335 27346 tablet_bootstrap.cc:654] T 7ff26678dfcf420a832e9ff714e507ee P ea6afc0ee252416cb5f2a31717dc3ac6: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:52.382582 27345 tablet_bootstrap.cc:654] T 7ff26678dfcf420a832e9ff714e507ee P c3dc2031d5af479ca10a5db2c3101eb5: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:52.383260 27348 tablet_bootstrap.cc:654] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:52.384039 27345 tablet_bootstrap.cc:492] T 7ff26678dfcf420a832e9ff714e507ee P c3dc2031d5af479ca10a5db2c3101eb5: No bootstrap required, opened a new log
I20260808 02:17:52.384133 27345 ts_tablet_manager.cc:1397] T 7ff26678dfcf420a832e9ff714e507ee P c3dc2031d5af479ca10a5db2c3101eb5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:17:52.384537 27346 tablet_bootstrap.cc:492] T 7ff26678dfcf420a832e9ff714e507ee P ea6afc0ee252416cb5f2a31717dc3ac6: No bootstrap required, opened a new log
I20260808 02:17:52.384629 27346 ts_tablet_manager.cc:1397] T 7ff26678dfcf420a832e9ff714e507ee P ea6afc0ee252416cb5f2a31717dc3ac6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:17:52.384649 27345 raft_consensus.cc:348] T 7ff26678dfcf420a832e9ff714e507ee P c3dc2031d5af479ca10a5db2c3101eb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } }
I20260808 02:17:52.384778 27345 raft_consensus.cc:374] T 7ff26678dfcf420a832e9ff714e507ee P c3dc2031d5af479ca10a5db2c3101eb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:52.384824 27345 raft_consensus.cc:729] T 7ff26678dfcf420a832e9ff714e507ee P c3dc2031d5af479ca10a5db2c3101eb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3dc2031d5af479ca10a5db2c3101eb5, State: Initialized, Role: FOLLOWER
I20260808 02:17:52.384965 27345 consensus_queue.cc:260] T 7ff26678dfcf420a832e9ff714e507ee P c3dc2031d5af479ca10a5db2c3101eb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } }
I20260808 02:17:52.385077 27346 raft_consensus.cc:348] T 7ff26678dfcf420a832e9ff714e507ee P ea6afc0ee252416cb5f2a31717dc3ac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } }
I20260808 02:17:52.385197 27346 raft_consensus.cc:374] T 7ff26678dfcf420a832e9ff714e507ee P ea6afc0ee252416cb5f2a31717dc3ac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:52.385231 27346 raft_consensus.cc:729] T 7ff26678dfcf420a832e9ff714e507ee P ea6afc0ee252416cb5f2a31717dc3ac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea6afc0ee252416cb5f2a31717dc3ac6, State: Initialized, Role: FOLLOWER
I20260808 02:17:52.385366 27346 consensus_queue.cc:260] T 7ff26678dfcf420a832e9ff714e507ee P ea6afc0ee252416cb5f2a31717dc3ac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } }
I20260808 02:17:52.385598 27348 tablet_bootstrap.cc:492] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d: No bootstrap required, opened a new log
I20260808 02:17:52.385627 27346 ts_tablet_manager.cc:1428] T 7ff26678dfcf420a832e9ff714e507ee P ea6afc0ee252416cb5f2a31717dc3ac6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 02:17:52.385679 27348 ts_tablet_manager.cc:1397] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 02:17:52.385926 27345 ts_tablet_manager.cc:1428] T 7ff26678dfcf420a832e9ff714e507ee P c3dc2031d5af479ca10a5db2c3101eb5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:17:52.386164 27348 raft_consensus.cc:348] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } }
I20260808 02:17:52.386273 27348 raft_consensus.cc:374] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:52.386307 27348 raft_consensus.cc:729] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9539d61446245fc8c8cc82470b7298d, State: Initialized, Role: FOLLOWER
I20260808 02:17:52.386426 27348 consensus_queue.cc:260] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } }
I20260808 02:17:52.386672 27348 ts_tablet_manager.cc:1428] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:17:52.418550 27355 raft_consensus.cc:482] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:17:52.418723 27355 raft_consensus.cc:504] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } }
I20260808 02:17:52.420161 27355 leader_election.cc:288] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3dc2031d5af479ca10a5db2c3101eb5 (127.24.79.193:41261), ea6afc0ee252416cb5f2a31717dc3ac6 (127.24.79.194:40929)
W20260808 02:17:52.423137 27308 tablet.cc:2369] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:17:52.425055 27336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.420945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:58589 (local address 127.24.79.194:40929)
0808 02:17:52.421072 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:52.421076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:52.421291 (+ 215us) server_negotiation.cc:407] Connection header received
0808 02:17:52.421389 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:52.421391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:52.421439 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:52.421515 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:52.421778 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.422440 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.424031 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.424443 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.424560 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.424622 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:52.424861 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 02:17:52.424941 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:17:52.425169 27146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.420374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:41261 (local address 127.24.79.195:56239)
0808 02:17:52.420559 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:52.420574 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:52.420659 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:52.421073 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:52.421076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:52.421096 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:52.421205 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.421211 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.422144 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.422148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.423636 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.423650 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.424709 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.424713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.424854 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.424879 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 02:17:52.424950 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 02:17:52.425024 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":26}
I20260808 02:17:52.425338 27343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.420586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:56239 (local address 127.24.79.193:41261)
0808 02:17:52.420734 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:52.420739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:52.420755 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:17:52.420815 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:52.420817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:52.420865 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:52.420941 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:52.421336 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.422034 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.423829 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.424470 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.424591 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.424645 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:52.425132 (+ 487us) server_negotiation.cc:299] Negotiation successful
0808 02:17:52.425215 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:17:52.425073 27381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.420548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:40929 (local address 127.24.79.195:58589)
0808 02:17:52.421172 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:52.421186 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:52.421300 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:52.421531 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:52.421533 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:52.421548 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:52.421677 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.421683 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.422546 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.422550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.423898 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.423908 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.424564 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.424567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.424731 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.424753 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 02:17:52.424822 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 02:17:52.424903 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"mutex_wait_us":55,"thread_start_us":90,"threads_started":1}
I20260808 02:17:52.425987 26820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ff26678dfcf420a832e9ff714e507ee" candidate_uuid: "d9539d61446245fc8c8cc82470b7298d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" is_pre_election: true
I20260808 02:17:52.426294 26820 raft_consensus.cc:2454] T 7ff26678dfcf420a832e9ff714e507ee P c3dc2031d5af479ca10a5db2c3101eb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9539d61446245fc8c8cc82470b7298d in term 0.
I20260808 02:17:52.426703 27028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ff26678dfcf420a832e9ff714e507ee" candidate_uuid: "d9539d61446245fc8c8cc82470b7298d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" is_pre_election: true
I20260808 02:17:52.426760 27137 leader_election.cc:302] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3dc2031d5af479ca10a5db2c3101eb5, d9539d61446245fc8c8cc82470b7298d; no voters:
I20260808 02:17:52.426867 27028 raft_consensus.cc:2454] T 7ff26678dfcf420a832e9ff714e507ee P ea6afc0ee252416cb5f2a31717dc3ac6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9539d61446245fc8c8cc82470b7298d in term 0.
I20260808 02:17:52.427095 27355 raft_consensus.cc:2781] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:17:52.427157 27355 raft_consensus.cc:482] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:17:52.427215 27355 raft_consensus.cc:3037] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:52.428285 27355 raft_consensus.cc:504] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } }
I20260808 02:17:52.428666 27355 leader_election.cc:288] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d [CANDIDATE]: Term 1 election: Requested vote from peers c3dc2031d5af479ca10a5db2c3101eb5 (127.24.79.193:41261), ea6afc0ee252416cb5f2a31717dc3ac6 (127.24.79.194:40929)
I20260808 02:17:52.429142 26820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ff26678dfcf420a832e9ff714e507ee" candidate_uuid: "d9539d61446245fc8c8cc82470b7298d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3dc2031d5af479ca10a5db2c3101eb5"
I20260808 02:17:52.429288 26820 raft_consensus.cc:3037] T 7ff26678dfcf420a832e9ff714e507ee P c3dc2031d5af479ca10a5db2c3101eb5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:52.430357 26820 raft_consensus.cc:2454] T 7ff26678dfcf420a832e9ff714e507ee P c3dc2031d5af479ca10a5db2c3101eb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9539d61446245fc8c8cc82470b7298d in term 1.
I20260808 02:17:52.430652 27137 leader_election.cc:302] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3dc2031d5af479ca10a5db2c3101eb5, d9539d61446245fc8c8cc82470b7298d; no voters:
I20260808 02:17:52.430635 27028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ff26678dfcf420a832e9ff714e507ee" candidate_uuid: "d9539d61446245fc8c8cc82470b7298d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6"
I20260808 02:17:52.430778 27028 raft_consensus.cc:3037] T 7ff26678dfcf420a832e9ff714e507ee P ea6afc0ee252416cb5f2a31717dc3ac6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:52.431764 27028 raft_consensus.cc:2454] T 7ff26678dfcf420a832e9ff714e507ee P ea6afc0ee252416cb5f2a31717dc3ac6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9539d61446245fc8c8cc82470b7298d in term 1.
I20260808 02:17:52.431986 27355 raft_consensus.cc:2781] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:52.432251 27355 raft_consensus.cc:686] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d [term 1 LEADER]: Becoming Leader. State: Replica: d9539d61446245fc8c8cc82470b7298d, State: Running, Role: LEADER
I20260808 02:17:52.432576 27355 consensus_queue.cc:237] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } }
W20260808 02:17:52.434747 27098 tablet.cc:2369] T 7ff26678dfcf420a832e9ff714e507ee P ea6afc0ee252416cb5f2a31717dc3ac6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:17:52.436017 26594 catalog_manager.cc:5495] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d reported cstate change: term changed from 0 to 1, leader changed from <none> to d9539d61446245fc8c8cc82470b7298d (127.24.79.195). New cstate: current_term: 1 leader_uuid: "d9539d61446245fc8c8cc82470b7298d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } health_report { overall_health: HEALTHY } } }
W20260808 02:17:52.442586 26884 tablet.cc:2369] T 7ff26678dfcf420a832e9ff714e507ee P c3dc2031d5af479ca10a5db2c3101eb5: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 02:17:52 dist-test-slave-1vmg krb5kdc[26526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155471, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:17:52.479825 27400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.469289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:42441 (local address 127.0.0.1:56736)
0808 02:17:52.469731 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:52.469747 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:52.469875 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:52.470231 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:52.470242 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:52.470820 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:52.470984 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.470993 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.472048 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.472051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.472653 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.472663 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.473253 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.473257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.473338 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.474041 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:52.474055 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:52.475846 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:52.477986 (+ 2140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:52.478001 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:52.478007 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:52.478315 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:52.478610 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:52.478613 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:52.478614 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:52.478706 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:52.479259 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:52.479266 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:52.479396 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 02:17:52.479505 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 02:17:52.479656 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":169,"threads_started":1}
I20260808 02:17:52.479955 26666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.469360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56736 (local address 127.24.79.254:42441)
0808 02:17:52.469510 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:52.469515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:52.469830 (+ 315us) server_negotiation.cc:407] Connection header received
0808 02:17:52.470002 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:52.470005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:52.470061 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:52.470158 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:52.471117 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.471921 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.472786 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.473117 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.473222 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.476022 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:52.476026 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:52.476029 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:52.476059 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:52.477836 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:52.478415 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:52.478418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:52.478420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:52.478518 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:52.478794 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:52.478798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:52.478799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:52.479106 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:52.479217 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:52.479593 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 02:17:52.479747 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:17:52.485682 26594 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:56736:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 02:17:52.486707 26594 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:17:52.492626 27206 tablet_service.cc:1467] Processing CreateTablet for tablet d43f505269864fc0975233f108404907 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=244112ea98754fe3b19f3304413c301c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:17:52.493151 27206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d43f505269864fc0975233f108404907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:52.492560 26793 tablet_service.cc:1467] Processing CreateTablet for tablet d43f505269864fc0975233f108404907 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=244112ea98754fe3b19f3304413c301c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:17:52.495450 26996 tablet_service.cc:1467] Processing CreateTablet for tablet d43f505269864fc0975233f108404907 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=244112ea98754fe3b19f3304413c301c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:17:52.495959 26996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d43f505269864fc0975233f108404907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:52.496320 27348 tablet_bootstrap.cc:492] T d43f505269864fc0975233f108404907 P d9539d61446245fc8c8cc82470b7298d: Bootstrap starting.
I20260808 02:17:52.497376 27348 tablet_bootstrap.cc:654] T d43f505269864fc0975233f108404907 P d9539d61446245fc8c8cc82470b7298d: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:52.497972 26793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d43f505269864fc0975233f108404907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:52.499466 27346 tablet_bootstrap.cc:492] T d43f505269864fc0975233f108404907 P ea6afc0ee252416cb5f2a31717dc3ac6: Bootstrap starting.
I20260808 02:17:52.500617 27346 tablet_bootstrap.cc:654] T d43f505269864fc0975233f108404907 P ea6afc0ee252416cb5f2a31717dc3ac6: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:52.501437 27345 tablet_bootstrap.cc:492] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5: Bootstrap starting.
I20260808 02:17:52.501813 27348 tablet_bootstrap.cc:492] T d43f505269864fc0975233f108404907 P d9539d61446245fc8c8cc82470b7298d: No bootstrap required, opened a new log
I20260808 02:17:52.501914 27348 ts_tablet_manager.cc:1397] T d43f505269864fc0975233f108404907 P d9539d61446245fc8c8cc82470b7298d: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260808 02:17:52.502079 27346 tablet_bootstrap.cc:492] T d43f505269864fc0975233f108404907 P ea6afc0ee252416cb5f2a31717dc3ac6: No bootstrap required, opened a new log
I20260808 02:17:52.502166 27346 ts_tablet_manager.cc:1397] T d43f505269864fc0975233f108404907 P ea6afc0ee252416cb5f2a31717dc3ac6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:17:52.502432 27348 raft_consensus.cc:348] T d43f505269864fc0975233f108404907 P d9539d61446245fc8c8cc82470b7298d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } }
I20260808 02:17:52.502548 27345 tablet_bootstrap.cc:654] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:52.502573 27348 raft_consensus.cc:374] T d43f505269864fc0975233f108404907 P d9539d61446245fc8c8cc82470b7298d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:52.502606 27348 raft_consensus.cc:729] T d43f505269864fc0975233f108404907 P d9539d61446245fc8c8cc82470b7298d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9539d61446245fc8c8cc82470b7298d, State: Initialized, Role: FOLLOWER
I20260808 02:17:52.502648 27346 raft_consensus.cc:348] T d43f505269864fc0975233f108404907 P ea6afc0ee252416cb5f2a31717dc3ac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } }
I20260808 02:17:52.502781 27346 raft_consensus.cc:374] T d43f505269864fc0975233f108404907 P ea6afc0ee252416cb5f2a31717dc3ac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:52.502763 27348 consensus_queue.cc:260] T d43f505269864fc0975233f108404907 P d9539d61446245fc8c8cc82470b7298d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } }
I20260808 02:17:52.502818 27346 raft_consensus.cc:729] T d43f505269864fc0975233f108404907 P ea6afc0ee252416cb5f2a31717dc3ac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea6afc0ee252416cb5f2a31717dc3ac6, State: Initialized, Role: FOLLOWER
I20260808 02:17:52.503098 27348 ts_tablet_manager.cc:1428] T d43f505269864fc0975233f108404907 P d9539d61446245fc8c8cc82470b7298d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:17:52.503048 27346 consensus_queue.cc:260] T d43f505269864fc0975233f108404907 P ea6afc0ee252416cb5f2a31717dc3ac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } }
I20260808 02:17:52.503362 27346 ts_tablet_manager.cc:1428] T d43f505269864fc0975233f108404907 P ea6afc0ee252416cb5f2a31717dc3ac6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:17:52.503347 27355 tablet_replica.cc:439] T d43f505269864fc0975233f108404907 P d9539d61446245fc8c8cc82470b7298d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } } }
I20260808 02:17:52.503492 27355 tablet_replica.cc:442] T d43f505269864fc0975233f108404907 P d9539d61446245fc8c8cc82470b7298d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:17:52.504087 27354 tablet_replica.cc:439] T d43f505269864fc0975233f108404907 P ea6afc0ee252416cb5f2a31717dc3ac6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } } }
I20260808 02:17:52.504244 27354 tablet_replica.cc:442] T d43f505269864fc0975233f108404907 P ea6afc0ee252416cb5f2a31717dc3ac6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:17:52.504598 27345 tablet_bootstrap.cc:492] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5: No bootstrap required, opened a new log
I20260808 02:17:52.504696 27345 ts_tablet_manager.cc:1397] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260808 02:17:52.505590 27345 raft_consensus.cc:348] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } }
I20260808 02:17:52.505717 27345 raft_consensus.cc:374] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:52.505750 27345 raft_consensus.cc:729] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3dc2031d5af479ca10a5db2c3101eb5, State: Initialized, Role: FOLLOWER
I20260808 02:17:52.505895 27345 consensus_queue.cc:260] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } }
I20260808 02:17:52.506170 27345 ts_tablet_manager.cc:1428] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260808 02:17:52.506404 27368 tablet_replica.cc:439] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } } }
I20260808 02:17:52.506573 27368 tablet_replica.cc:442] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:17:52.522285 27368 raft_consensus.cc:482] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:17:52.522442 27368 raft_consensus.cc:504] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } }
I20260808 02:17:52.522974 27368 leader_election.cc:288] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea6afc0ee252416cb5f2a31717dc3ac6 (127.24.79.194:40929), d9539d61446245fc8c8cc82470b7298d (127.24.79.195:37987)
I20260808 02:17:52.523520 27028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43f505269864fc0975233f108404907" candidate_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" is_pre_election: true
I20260808 02:17:52.523530 27237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43f505269864fc0975233f108404907" candidate_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9539d61446245fc8c8cc82470b7298d" is_pre_election: true
I20260808 02:17:52.523697 27028 raft_consensus.cc:2454] T d43f505269864fc0975233f108404907 P ea6afc0ee252416cb5f2a31717dc3ac6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3dc2031d5af479ca10a5db2c3101eb5 in term 0.
I20260808 02:17:52.523710 27237 raft_consensus.cc:2454] T d43f505269864fc0975233f108404907 P d9539d61446245fc8c8cc82470b7298d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3dc2031d5af479ca10a5db2c3101eb5 in term 0.
I20260808 02:17:52.524083 26720 leader_election.cc:302] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3dc2031d5af479ca10a5db2c3101eb5, ea6afc0ee252416cb5f2a31717dc3ac6; no voters:
I20260808 02:17:52.524322 27368 raft_consensus.cc:2781] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:17:52.524375 27368 raft_consensus.cc:482] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:17:52.524413 27368 raft_consensus.cc:3037] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:52.525449 27368 raft_consensus.cc:504] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } }
I20260808 02:17:52.525836 27368 leader_election.cc:288] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5 [CANDIDATE]: Term 1 election: Requested vote from peers ea6afc0ee252416cb5f2a31717dc3ac6 (127.24.79.194:40929), d9539d61446245fc8c8cc82470b7298d (127.24.79.195:37987)
I20260808 02:17:52.526288 27028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43f505269864fc0975233f108404907" candidate_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6"
I20260808 02:17:52.526300 27237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43f505269864fc0975233f108404907" candidate_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9539d61446245fc8c8cc82470b7298d"
I20260808 02:17:52.526423 27028 raft_consensus.cc:3037] T d43f505269864fc0975233f108404907 P ea6afc0ee252416cb5f2a31717dc3ac6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:52.526443 27237 raft_consensus.cc:3037] T d43f505269864fc0975233f108404907 P d9539d61446245fc8c8cc82470b7298d [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:52.527483 27028 raft_consensus.cc:2454] T d43f505269864fc0975233f108404907 P ea6afc0ee252416cb5f2a31717dc3ac6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3dc2031d5af479ca10a5db2c3101eb5 in term 1.
I20260808 02:17:52.527767 27237 raft_consensus.cc:2454] T d43f505269864fc0975233f108404907 P d9539d61446245fc8c8cc82470b7298d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3dc2031d5af479ca10a5db2c3101eb5 in term 1.
I20260808 02:17:52.527872 26720 leader_election.cc:302] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3dc2031d5af479ca10a5db2c3101eb5, ea6afc0ee252416cb5f2a31717dc3ac6; no voters:
I20260808 02:17:52.528110 27368 raft_consensus.cc:2781] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:52.528205 27368 raft_consensus.cc:686] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5 [term 1 LEADER]: Becoming Leader. State: Replica: c3dc2031d5af479ca10a5db2c3101eb5, State: Running, Role: LEADER
I20260808 02:17:52.528348 27368 consensus_queue.cc:237] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } }
I20260808 02:17:52.529192 27353 tablet_replica.cc:439] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5: TxnStatusTablet state changed. Reason: New leader c3dc2031d5af479ca10a5db2c3101eb5. Latest consensus state: current_term: 1 leader_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } } }
I20260808 02:17:52.529371 27353 tablet_replica.cc:442] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:17:52.529975 27408 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 02:17:52.530190 27408 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 02:17:52.530444 26591 catalog_manager.cc:5495] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3dc2031d5af479ca10a5db2c3101eb5 (127.24.79.193). New cstate: current_term: 1 leader_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } health_report { overall_health: UNKNOWN } } }
I20260808 02:17:52.551318 27400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.547415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:41261 (local address 127.0.0.1:47128)
0808 02:17:52.547583 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:52.547600 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:52.547701 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:52.548003 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:52.548005 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:52.548012 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:17:52.548165 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.548170 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.549207 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.549209 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.549838 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.549847 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.550440 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.550444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.550527 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.550594 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:17:52.550987 (+ 393us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:17:52.551013 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 02:17:52.551089 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 02:17:52.551181 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 02:17:52.551329 27343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.547542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47128 (local address 127.24.79.193:41261)
0808 02:17:52.547674 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:52.547679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:52.547696 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:17:52.547818 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:52.547821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:52.547875 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:52.547954 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:52.548265 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.549078 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.549951 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.550302 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.550407 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.550744 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:52.550844 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:52.550938 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:52.551105 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 02:17:52.551176 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 02:17:52.554549 27028 raft_consensus.cc:1261] T d43f505269864fc0975233f108404907 P ea6afc0ee252416cb5f2a31717dc3ac6 [term 1 FOLLOWER]: Refusing update from remote peer c3dc2031d5af479ca10a5db2c3101eb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:52.554620 27237 raft_consensus.cc:1261] T d43f505269864fc0975233f108404907 P d9539d61446245fc8c8cc82470b7298d [term 1 FOLLOWER]: Refusing update from remote peer c3dc2031d5af479ca10a5db2c3101eb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:52.555111 27353 consensus_queue.cc:1035] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:52.555295 27368 consensus_queue.cc:1035] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:52.556846 27355 tablet_replica.cc:439] T d43f505269864fc0975233f108404907 P d9539d61446245fc8c8cc82470b7298d: TxnStatusTablet state changed. Reason: New leader c3dc2031d5af479ca10a5db2c3101eb5. Latest consensus state: current_term: 1 leader_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } } }
I20260808 02:17:52.556872 27354 tablet_replica.cc:439] T d43f505269864fc0975233f108404907 P ea6afc0ee252416cb5f2a31717dc3ac6: TxnStatusTablet state changed. Reason: New leader c3dc2031d5af479ca10a5db2c3101eb5. Latest consensus state: current_term: 1 leader_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } } }
I20260808 02:17:52.556985 27355 tablet_replica.cc:442] T d43f505269864fc0975233f108404907 P d9539d61446245fc8c8cc82470b7298d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:17:52.556998 27354 tablet_replica.cc:442] T d43f505269864fc0975233f108404907 P ea6afc0ee252416cb5f2a31717dc3ac6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:17:52.558799 27353 tablet_replica.cc:439] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } } }
I20260808 02:17:52.558955 27353 tablet_replica.cc:442] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:17:52.559067 27363 mvcc.cc:205] Tried to move back new op lower bound from 7316092815578783744 to 7316092815477841920. Current Snapshot: MvccSnapshot[applied={T|T < 7316092815578783744 or (T in {7316092815578783744})}]
I20260808 02:17:52.560434 27355 tablet_replica.cc:439] T d43f505269864fc0975233f108404907 P d9539d61446245fc8c8cc82470b7298d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } } }
I20260808 02:17:52.560560 27355 tablet_replica.cc:442] T d43f505269864fc0975233f108404907 P d9539d61446245fc8c8cc82470b7298d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:17:52.562335 27368 tablet_replica.cc:439] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } } }
I20260808 02:17:52.562477 27368 tablet_replica.cc:442] T d43f505269864fc0975233f108404907 P c3dc2031d5af479ca10a5db2c3101eb5: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:17:52.563551 27354 tablet_replica.cc:439] T d43f505269864fc0975233f108404907 P ea6afc0ee252416cb5f2a31717dc3ac6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 } } peers { permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 } } peers { permanent_uuid: "d9539d61446245fc8c8cc82470b7298d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37987 } } }
I20260808 02:17:52.563683 27354 tablet_replica.cc:442] T d43f505269864fc0975233f108404907 P ea6afc0ee252416cb5f2a31717dc3ac6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:17:52.570446 27340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.566148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39840 (local address 127.24.79.195:37987)
0808 02:17:52.566590 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:52.566594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:52.566605 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:17:52.566680 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:52.566682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:52.566735 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:52.566817 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:52.567133 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.567645 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.568822 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.569092 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.569195 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.569485 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:52.570041 (+ 556us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:52.570151 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:52.570227 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 02:17:52.570283 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352}
I20260808 02:17:52.582039 27343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.578778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:56213 (local address 127.24.79.193:41261)
0808 02:17:52.578943 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:52.578949 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:52.578965 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:17:52.579140 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:52.579143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:52.579194 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:52.579258 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:52.579544 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.580074 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.580944 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.581166 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.581271 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.581584 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:52.581664 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:52.581725 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:52.581829 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 02:17:52.581889 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:17:52.582039 27313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.578637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:41261 (local address 127.24.79.195:56213)
0808 02:17:52.578866 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:52.578881 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:52.579030 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:52.579288 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:52.579291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:52.579299 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:17:52.579434 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.579440 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.580188 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.580192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.580822 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.580831 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.581287 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.581291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.581390 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.581440 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:17:52.581742 (+ 302us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:17:52.581749 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 02:17:52.581817 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 02:17:52.581900 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121}
I20260808 02:17:52.590126 27028 raft_consensus.cc:1261] T 7ff26678dfcf420a832e9ff714e507ee P ea6afc0ee252416cb5f2a31717dc3ac6 [term 1 FOLLOWER]: Refusing update from remote peer d9539d61446245fc8c8cc82470b7298d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:52.590355 26820 raft_consensus.cc:1261] T 7ff26678dfcf420a832e9ff714e507ee P c3dc2031d5af479ca10a5db2c3101eb5 [term 1 FOLLOWER]: Refusing update from remote peer d9539d61446245fc8c8cc82470b7298d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:52.590785 27387 consensus_queue.cc:1035] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6afc0ee252416cb5f2a31717dc3ac6" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 40929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:52.591154 27355 consensus_queue.cc:1035] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3dc2031d5af479ca10a5db2c3101eb5" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:52.595882 27365 mvcc.cc:205] Tried to move back new op lower bound from 7316092815724175360 to 7316092815086796800. Current Snapshot: MvccSnapshot[applied={T|T < 7316092815724175360}]
I20260808 02:17:52.616902 27340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.613398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:51749 (local address 127.24.79.195:37987)
0808 02:17:52.613518 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:52.613523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:52.613538 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:17:52.613601 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:52.613604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:52.613651 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:52.613710 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:52.613960 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.614547 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.615499 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.615800 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.615896 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.616248 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:52.616352 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:52.616429 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:52.616675 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 02:17:52.616747 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 02:17:52.617540 27421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.612988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:37987 (local address 127.24.79.193:51749)
0808 02:17:52.613352 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:52.613368 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:52.613479 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:52.613736 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:52.613739 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:52.613747 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:17:52.613859 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.613866 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.614666 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.614670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.615374 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:52.615385 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.615919 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.615922 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:52.616008 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.616083 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:17:52.616473 (+ 390us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:17:52.616481 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 02:17:52.617284 (+ 803us) client_negotiation.cc:240] Negotiation successful
0808 02:17:52.617384 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":104,"threads_started":1}
I20260808 02:17:52.649786 26594 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:17:52.649999 26594 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:17:52.652721 26594 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 811b94b7fcdf4dfda3fe111460c0048b: Sending DeleteTablet for 3 replicas of tablet 7ff26678dfcf420a832e9ff714e507ee
I20260808 02:17:52.653755 26996 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ff26678dfcf420a832e9ff714e507ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:52 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:60538
I20260808 02:17:52.653908 27206 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ff26678dfcf420a832e9ff714e507ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:52 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:39836
I20260808 02:17:52.654156 27375 tablet_replica.cc:330] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d: stopping tablet replica
I20260808 02:17:52.654163 27374 tablet_replica.cc:330] T 7ff26678dfcf420a832e9ff714e507ee P ea6afc0ee252416cb5f2a31717dc3ac6: stopping tablet replica
I20260808 02:17:52.654297 27375 raft_consensus.cc:2229] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:17:52.654299 27374 raft_consensus.cc:2229] T 7ff26678dfcf420a832e9ff714e507ee P ea6afc0ee252416cb5f2a31717dc3ac6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:17:52.654438 27374 raft_consensus.cc:2258] T 7ff26678dfcf420a832e9ff714e507ee P ea6afc0ee252416cb5f2a31717dc3ac6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:52.654659 26793 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ff26678dfcf420a832e9ff714e507ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:52 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:47110
I20260808 02:17:52.654685 27375 raft_consensus.cc:2258] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:52.654862 27376 tablet_replica.cc:330] T 7ff26678dfcf420a832e9ff714e507ee P c3dc2031d5af479ca10a5db2c3101eb5: stopping tablet replica
I20260808 02:17:52.655030 27376 raft_consensus.cc:2229] T 7ff26678dfcf420a832e9ff714e507ee P c3dc2031d5af479ca10a5db2c3101eb5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:17:52.655167 27376 raft_consensus.cc:2258] T 7ff26678dfcf420a832e9ff714e507ee P c3dc2031d5af479ca10a5db2c3101eb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:52.655634 27375 ts_tablet_manager.cc:1905] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:17:52.656369 27376 ts_tablet_manager.cc:1905] T 7ff26678dfcf420a832e9ff714e507ee P c3dc2031d5af479ca10a5db2c3101eb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:17:52.659021 27375 ts_tablet_manager.cc:1918] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:17:52.659107 27375 log.cc:1196] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/7ff26678dfcf420a832e9ff714e507ee
I20260808 02:17:52.659371 27375 ts_tablet_manager.cc:1939] T 7ff26678dfcf420a832e9ff714e507ee P d9539d61446245fc8c8cc82470b7298d: Deleting consensus metadata
I20260808 02:17:52.659441 27376 ts_tablet_manager.cc:1918] T 7ff26678dfcf420a832e9ff714e507ee P c3dc2031d5af479ca10a5db2c3101eb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:17:52.659521 27376 log.cc:1196] T 7ff26678dfcf420a832e9ff714e507ee P c3dc2031d5af479ca10a5db2c3101eb5: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-0/wal/wals/7ff26678dfcf420a832e9ff714e507ee
I20260808 02:17:52.659795 27376 ts_tablet_manager.cc:1939] T 7ff26678dfcf420a832e9ff714e507ee P c3dc2031d5af479ca10a5db2c3101eb5: Deleting consensus metadata
I20260808 02:17:52.660315 26573 catalog_manager.cc:4841] TS d9539d61446245fc8c8cc82470b7298d (127.24.79.195:37987): tablet 7ff26678dfcf420a832e9ff714e507ee (table test-table [id=8edc4082e2f646d1837ee985c4ee95a1]) successfully deleted
I20260808 02:17:52.660853 26574 catalog_manager.cc:4841] TS c3dc2031d5af479ca10a5db2c3101eb5 (127.24.79.193:41261): tablet 7ff26678dfcf420a832e9ff714e507ee (table test-table [id=8edc4082e2f646d1837ee985c4ee95a1]) successfully deleted
I20260808 02:17:52.661751 27374 ts_tablet_manager.cc:1905] T 7ff26678dfcf420a832e9ff714e507ee P ea6afc0ee252416cb5f2a31717dc3ac6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:17:52.665103 27374 ts_tablet_manager.cc:1918] T 7ff26678dfcf420a832e9ff714e507ee P ea6afc0ee252416cb5f2a31717dc3ac6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:17:52.665211 27374 log.cc:1196] T 7ff26678dfcf420a832e9ff714e507ee P ea6afc0ee252416cb5f2a31717dc3ac6: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/7ff26678dfcf420a832e9ff714e507ee
I20260808 02:17:52.665465 27374 ts_tablet_manager.cc:1939] T 7ff26678dfcf420a832e9ff714e507ee P ea6afc0ee252416cb5f2a31717dc3ac6: Deleting consensus metadata
I20260808 02:17:52.666466 26570 catalog_manager.cc:4841] TS ea6afc0ee252416cb5f2a31717dc3ac6 (127.24.79.194:40929): tablet 7ff26678dfcf420a832e9ff714e507ee (table test-table [id=8edc4082e2f646d1837ee985c4ee95a1]) successfully deleted
Aug 08 02:17:52 dist-test-slave-1vmg krb5kdc[26526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155472, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.79.193@KRBTEST.COM
I20260808 02:17:52.680603 27343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.663380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47130 (local address 127.24.79.193:41261)
0808 02:17:52.663518 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:52.663523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:52.666522 (+ 2999us) server_negotiation.cc:407] Connection header received
0808 02:17:52.666779 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:52.666783 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:52.666840 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:52.666897 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:52.668012 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.668858 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.670188 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.670532 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.670672 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.675283 (+ 4611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:52.675305 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:52.675329 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:52.675372 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:52.677768 (+ 2396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:52.678495 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:52.678504 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:52.678506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:52.678655 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:52.679290 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:52.679293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:52.679295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:52.679695 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:52.679970 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:52.680222 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 02:17:52.680397 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 02:17:52.681869 26875 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47130
I20260808 02:17:52.697297 26666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:52.687484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56752 (local address 127.24.79.254:42441)
0808 02:17:52.687640 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:52.687645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:52.687926 (+ 281us) server_negotiation.cc:407] Connection header received
0808 02:17:52.688056 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:52.688059 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:52.688121 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:52.688210 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:52.689074 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.689961 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.691269 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:52.691602 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:52.691695 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:52.693629 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:52.693634 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:52.693636 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:52.693674 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:52.695278 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:52.695783 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:52.695786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:52.695788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:52.695884 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:52.696176 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:52.696179 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:52.696180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:52.696433 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:52.696534 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:52.696967 (+ 433us) server_negotiation.cc:299] Negotiation successful
0808 02:17:52.697104 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 02:17:52.698617 26594 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56752
I20260808 02:17:52.700117 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 26702
I20260808 02:17:52.713726 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 26898
I20260808 02:17:52.735152 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 27109
I20260808 02:17:52.749799 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 26543
2026-08-08T02:17:52Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2844 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:52 dist-test-slave-1vmg krb5kdc[27437](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:52 dist-test-slave-1vmg krb5kdc[27437](info): set up 2 sockets
Aug 08 02:17:52 dist-test-slave-1vmg krb5kdc[27437](info): commencing operation
krb5kdc: starting...
W20260808 02:17:53.910735 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:17:54 dist-test-slave-1vmg krb5kdc[27437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:17:54.058116 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:32993
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:32993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:54.179518 27458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:54.179878 27458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:54.179945 27458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:54.184104 27458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:54.184187 27458 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 02:17:54.184211 27458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:54.184235 27458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:54.184257 27458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:54.184283 27458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:54.190120 27458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:32993
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:32993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:54.191397 27458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:54.192503 27458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:54.201932 27469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:54.201920 27467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:54.201920 27465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:54.202325 27458 server_base.cc:1034] running on GCE node
I20260808 02:17:54.202983 27458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:54.204253 27458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:54.205466 27458 hybrid_clock.cc:648] HybridClock initialized: now 1786155474205439 us; error 39 us; skew 500 ppm
Aug 08 02:17:54 dist-test-slave-1vmg krb5kdc[27437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155474, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:54.208663 27458 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:17:54.209712 27458 webserver.cc:460] Webserver started at http://127.24.79.254:33673/ using document root <none> and password file <none>
I20260808 02:17:54.210314 27458 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:54.210386 27458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:54.210601 27458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:54.212865 27458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "f7632fff81f5499a898c713ff58442d1"
format_stamp: "Formatted at 2026-08-08 02:17:54 on dist-test-slave-1vmg"
server_key: "8037789bdfe75ab91d6680778d284580"
server_key_iv: "d767845edfe403417f3384e1b9a7318e"
server_key_version: "encryptionkey@0"
I20260808 02:17:54.213455 27458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "f7632fff81f5499a898c713ff58442d1"
format_stamp: "Formatted at 2026-08-08 02:17:54 on dist-test-slave-1vmg"
server_key: "8037789bdfe75ab91d6680778d284580"
server_key_iv: "d767845edfe403417f3384e1b9a7318e"
server_key_version: "encryptionkey@0"
I20260808 02:17:54.217923 27458 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 02:17:54.221066 27477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:54.222584 27458 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:17:54.222746 27458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "f7632fff81f5499a898c713ff58442d1"
format_stamp: "Formatted at 2026-08-08 02:17:54 on dist-test-slave-1vmg"
server_key: "8037789bdfe75ab91d6680778d284580"
server_key_iv: "d767845edfe403417f3384e1b9a7318e"
server_key_version: "encryptionkey@0"
I20260808 02:17:54.222877 27458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:54.236311 27458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:54.240100 27458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:54.240335 27458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:54.249766 27458 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:32993
I20260808 02:17:54.249811 27573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:32993 every 8 connection(s)
I20260808 02:17:54.250949 27458 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:54.254186 27574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:54.256623 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 27458
I20260808 02:17:54.256752 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:17:54.257026 24895 external_mini_cluster.cc:1428] Setting key aa1d52b1f5cd7093374caa5da7026faa
I20260808 02:17:54.261292 27574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1: Bootstrap starting.
I20260808 02:17:54.264122 27574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:54.264976 27574 log.cc:824] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:17:54 dist-test-slave-1vmg krb5kdc[27437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:17:54.267446 27574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1: No bootstrap required, opened a new log
I20260808 02:17:54.270535 27574 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7632fff81f5499a898c713ff58442d1" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 32993 } }
I20260808 02:17:54.270802 27574 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:54.270866 27574 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7632fff81f5499a898c713ff58442d1, State: Initialized, Role: FOLLOWER
I20260808 02:17:54.271337 27574 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7632fff81f5499a898c713ff58442d1" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 32993 } }
I20260808 02:17:54.271478 27574 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:54.271546 27574 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:54.271659 27574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:54.272783 27574 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7632fff81f5499a898c713ff58442d1" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 32993 } }
I20260808 02:17:54.273134 27574 leader_election.cc:302] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7632fff81f5499a898c713ff58442d1; no voters:
I20260808 02:17:54.273278 27578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:54.258746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53550 (local address 127.24.79.254:32993)
0808 02:17:54.259322 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:54.259339 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:54.259542 (+ 203us) server_negotiation.cc:407] Connection header received
0808 02:17:54.260361 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:54.260382 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:54.260753 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:54.261059 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:54.261991 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:54.263109 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:54.264312 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:54.264806 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:54.264929 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:54.267836 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:54.267842 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:54.267857 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:54.267887 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:54.270125 (+ 2238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:54.270597 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:54.270603 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:54.270608 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:54.270737 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:54.271067 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:54.271070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:54.271072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:54.271566 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:54.271779 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:54.272070 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 02:17:54.272433 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":194,"threads_started":1}
I20260808 02:17:54.273459 27574 leader_election.cc:288] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:54.273721 27580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:54.273941 27580 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1 [term 1 LEADER]: Becoming Leader. State: Replica: f7632fff81f5499a898c713ff58442d1, State: Running, Role: LEADER
I20260808 02:17:54.274260 27580 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7632fff81f5499a898c713ff58442d1" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 32993 } }
I20260808 02:17:54.274784 27574 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:54.276916 27583 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7632fff81f5499a898c713ff58442d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7632fff81f5499a898c713ff58442d1" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 32993 } } }
I20260808 02:17:54.277064 27583 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:54.277343 27584 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7632fff81f5499a898c713ff58442d1. Latest consensus state: current_term: 1 leader_uuid: "f7632fff81f5499a898c713ff58442d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7632fff81f5499a898c713ff58442d1" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 32993 } } }
I20260808 02:17:54.277760 27584 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1 [sys.catalog]: This master's current role is: LEADER
W20260808 02:17:54.280321 27609 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:17:54.280426 27609 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:17:54.280514 27608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:54.281451 27608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:54.287909 27608 catalog_manager.cc:1347] Generated new cluster ID: c9019ae0c93047cfbb0ae7d716998651
I20260808 02:17:54.287988 27608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:54.315593 27608 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:54.316850 27608 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:54.323933 27608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1: Generated new TSK 0
I20260808 02:17:54.324870 27608 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.79.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
I20260808 02:17:54.358268 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:32993
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:54.479915 27615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:54.480217 27615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:54.480265 27615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:54.484467 27615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:54.484548 27615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:54.484596 27615 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 02:17:54.484611 27615 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:17:54.484651 27615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:17:54.490295 27615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.79.254:32993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:54.491621 27615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:54.492762 27615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:54.504331 27624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:54.504335 27621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:54.504491 27615 server_base.cc:1034] running on GCE node
W20260808 02:17:54.504335 27622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:54.505285 27615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:54.506075 27615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:54.507285 27615 hybrid_clock.cc:648] HybridClock initialized: now 1786155474507263 us; error 40 us; skew 500 ppm
Aug 08 02:17:54 dist-test-slave-1vmg krb5kdc[27437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155474, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:54.510514 27615 init.cc:377] Logged in from keytab as kudu/127.24.79.193@KRBTEST.COM (short username kudu)
I20260808 02:17:54.511700 27615 webserver.cc:460] Webserver started at http://127.24.79.193:40163/ using document root <none> and password file <none>
I20260808 02:17:54.512508 27615 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:54.512601 27615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:54.512853 27615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:54.515353 27615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "8aed7f68aa194ebba090205fee848e85"
format_stamp: "Formatted at 2026-08-08 02:17:54 on dist-test-slave-1vmg"
server_key: "92facd159120c7d9e94e887ab9df1bb1"
server_key_iv: "8a5b689eb48297337b1341a5bc8bf083"
server_key_version: "encryptionkey@0"
I20260808 02:17:54.515993 27615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "8aed7f68aa194ebba090205fee848e85"
format_stamp: "Formatted at 2026-08-08 02:17:54 on dist-test-slave-1vmg"
server_key: "92facd159120c7d9e94e887ab9df1bb1"
server_key_iv: "8a5b689eb48297337b1341a5bc8bf083"
server_key_version: "encryptionkey@0"
I20260808 02:17:54.520467 27615 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:17:54.523459 27639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:54.525005 27615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:17:54.525168 27615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "8aed7f68aa194ebba090205fee848e85"
format_stamp: "Formatted at 2026-08-08 02:17:54 on dist-test-slave-1vmg"
server_key: "92facd159120c7d9e94e887ab9df1bb1"
server_key_iv: "8a5b689eb48297337b1341a5bc8bf083"
server_key_version: "encryptionkey@0"
I20260808 02:17:54.525314 27615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:54.536818 27615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:54.540489 27615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:54.540710 27615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:54.542487 27615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:17:54.542550 27615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:54.542613 27615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:17:54.542641 27615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:54.554061 27615 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:45371
Aug 08 02:17:54 dist-test-slave-1vmg krb5kdc[27437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155474, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:17:54.554291 27808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:45371 every 8 connection(s)
I20260808 02:17:54.555195 27615 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:17:54.556500 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 27615
I20260808 02:17:54.556628 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:17:54.556933 24895 external_mini_cluster.cc:1428] Setting key b8d0e73fbb0aedf3c364a25093f5319b
I20260808 02:17:54.559036 27578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:54.543586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:53443 (local address 127.24.79.254:32993)
0808 02:17:54.543733 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:54.543736 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:54.544924 (+ 1188us) server_negotiation.cc:407] Connection header received
0808 02:17:54.546271 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:54.546276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:54.546348 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:54.546457 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:54.548386 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:54.548979 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:54.550839 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:54.551159 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:54.551378 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:54.554897 (+ 3519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:54.554900 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:54.554902 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:54.554978 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:54.556649 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:54.557245 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:54.557249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:54.557251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:54.557365 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:54.557801 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:54.557804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:54.557806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:54.558052 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:54.558140 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:54.558688 (+ 548us) server_negotiation.cc:299] Negotiation successful
0808 02:17:54.558832 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":57}
I20260808 02:17:54.559968 27656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:54.544141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:32993 (local address 127.24.79.193:53443)
0808 02:17:54.544733 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:54.544785 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:54.545922 (+ 1137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:54.546707 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:54.546716 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:54.547300 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:54.548136 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:54.548158 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:54.549118 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:54.549122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:54.550671 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:54.550684 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:54.551423 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:54.551429 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:54.551812 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:54.552707 (+ 895us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:54.552745 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:54.554666 (+ 1921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:54.556770 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:54.556775 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:54.556788 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:54.557109 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:54.557477 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:54.557481 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:54.557483 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:54.557715 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:54.558159 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:54.558167 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:54.558443 (+ 276us) client_negotiation.cc:769] Sending connection context
0808 02:17:54.558625 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 02:17:54.559042 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":151,"threads_started":1}
I20260808 02:17:54.561542 27809 heartbeater.cc:347] Connected to a master server at 127.24.79.254:32993
I20260808 02:17:54.561745 27809 heartbeater.cc:464] Registering TS with master...
I20260808 02:17:54.562423 27809 heartbeater.cc:511] Master 127.24.79.254:32993 requested a full tablet report, sending...
I20260808 02:17:54.564419 27500 ts_manager.cc:194] Registered new tserver with Master: 8aed7f68aa194ebba090205fee848e85 (127.24.79.193:45371)
I20260808 02:17:54.565635 27500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.193@KRBTEST.COM'} at 127.24.79.193:53443
WARNING: no policy specified for kudu/127.24.79.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.194@KRBTEST.COM" created.
I20260808 02:17:54.574347 27578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:54.564058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:44015 (local address 127.24.79.254:32993)
0808 02:17:54.564179 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:54.564183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:54.567268 (+ 3085us) server_negotiation.cc:407] Connection header received
0808 02:17:54.567410 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:54.567412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:54.567453 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:54.567518 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:54.567997 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:54.568512 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:54.569362 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:54.569634 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:54.569748 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:54.570994 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:54.570997 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:54.570999 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:54.571026 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:54.572627 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:54.573076 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:54.573080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:54.573082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:54.573181 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:54.573425 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:54.573428 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:54.573430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:54.573676 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:54.573763 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:54.574041 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 02:17:54.574156 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 02:17:54.575119 27820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:54.563971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:32993 (local address 127.24.79.193:44015)
0808 02:17:54.567147 (+ 3176us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:54.567160 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:54.567317 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:54.567551 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:54.567555 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:54.567774 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:54.567879 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:54.567886 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:54.568660 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:54.568664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:54.569224 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:54.569233 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:54.569785 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:54.569788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:54.569862 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:54.570392 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:54.570404 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:54.570816 (+ 412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:54.572760 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:54.572763 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:54.572765 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:54.572981 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:54.573265 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:54.573268 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:54.573270 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:54.573349 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:54.573773 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:54.573776 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:54.573890 (+ 114us) client_negotiation.cc:769] Sending connection context
0808 02:17:54.574001 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 02:17:54.574214 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3079,"thread_start_us":126,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
I20260808 02:17:54.592191 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:32993
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:54.715234 27822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:54.715564 27822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:54.715615 27822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:54.719944 27822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:54.720032 27822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:54.720077 27822 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 02:17:54.720089 27822 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:17:54.720130 27822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:17:54.726300 27822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.79.254:32993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:54.727728 27822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:54.728902 27822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:54.739713 27832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:54.739811 27822 server_base.cc:1034] running on GCE node
W20260808 02:17:54.739737 27829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:54.739713 27828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:54.740665 27822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:54.741534 27822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:54.742784 27822 hybrid_clock.cc:648] HybridClock initialized: now 1786155474742748 us; error 55 us; skew 500 ppm
Aug 08 02:17:54 dist-test-slave-1vmg krb5kdc[27437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155474, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:54.745957 27822 init.cc:377] Logged in from keytab as kudu/127.24.79.194@KRBTEST.COM (short username kudu)
I20260808 02:17:54.747052 27822 webserver.cc:460] Webserver started at http://127.24.79.194:38151/ using document root <none> and password file <none>
I20260808 02:17:54.747705 27822 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:54.747779 27822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:54.747979 27822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:54.750355 27822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "73a70177480547aebc6eddbbde64eefd"
format_stamp: "Formatted at 2026-08-08 02:17:54 on dist-test-slave-1vmg"
server_key: "90f771b1dddc3d12872db1a1f466d4da"
server_key_iv: "ed12f33a312edcf886f16474f3aa958e"
server_key_version: "encryptionkey@0"
I20260808 02:17:54.751050 27822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "73a70177480547aebc6eddbbde64eefd"
format_stamp: "Formatted at 2026-08-08 02:17:54 on dist-test-slave-1vmg"
server_key: "90f771b1dddc3d12872db1a1f466d4da"
server_key_iv: "ed12f33a312edcf886f16474f3aa958e"
server_key_version: "encryptionkey@0"
I20260808 02:17:54.755501 27822 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:17:54.758605 27841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:54.760216 27822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:17:54.760398 27822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "73a70177480547aebc6eddbbde64eefd"
format_stamp: "Formatted at 2026-08-08 02:17:54 on dist-test-slave-1vmg"
server_key: "90f771b1dddc3d12872db1a1f466d4da"
server_key_iv: "ed12f33a312edcf886f16474f3aa958e"
server_key_version: "encryptionkey@0"
I20260808 02:17:54.760545 27822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:54.793900 27822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:54.800961 27822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:54.801199 27822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:54.802881 27822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:17:54.802981 27822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:54.803038 27822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:17:54.803067 27822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:17:54 dist-test-slave-1vmg krb5kdc[27437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155474, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:17:54.813704 27822 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:46423
I20260808 02:17:54.813768 28017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:46423 every 8 connection(s)
I20260808 02:17:54.814677 27822 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:17:54.817324 27578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:54.803956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:40087 (local address 127.24.79.254:32993)
0808 02:17:54.804141 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:54.804146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:54.805057 (+ 911us) server_negotiation.cc:407] Connection header received
0808 02:17:54.806313 (+ 1256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:54.806318 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:54.806385 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:54.806448 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:54.808184 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:54.808760 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:54.810064 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:54.810295 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:54.810391 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:54.813439 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:54.813444 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:54.813446 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:54.813484 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:54.815046 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:54.815671 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:54.815675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:54.815677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:54.815774 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:54.816121 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:54.816125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:54.816127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:54.816350 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:54.816442 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:54.816991 (+ 549us) server_negotiation.cc:299] Negotiation successful
0808 02:17:54.817137 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:17:54.818178 27855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:54.804360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:32993 (local address 127.24.79.194:40087)
0808 02:17:54.804872 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:54.804928 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:54.806008 (+ 1080us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:54.806881 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:54.806891 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:54.807456 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:54.808011 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:54.808027 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:54.808898 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:54.808902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:54.809924 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:54.809936 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:54.810459 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:54.810463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:54.810648 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:54.811395 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:54.811414 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:54.813222 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:54.815201 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:54.815207 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:54.815224 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:54.815571 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:54.815867 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:54.815871 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:54.815873 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:54.816022 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:54.816460 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:54.816467 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:54.816743 (+ 276us) client_negotiation.cc:769] Sending connection context
0808 02:17:54.816925 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 02:17:54.817288 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":124,"threads_started":1}
I20260808 02:17:54.819577 28018 heartbeater.cc:347] Connected to a master server at 127.24.79.254:32993
I20260808 02:17:54.819792 28018 heartbeater.cc:464] Registering TS with master...
I20260808 02:17:54.820467 28018 heartbeater.cc:511] Master 127.24.79.254:32993 requested a full tablet report, sending...
I20260808 02:17:54.821616 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 27822
I20260808 02:17:54.821736 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:17:54.821902 27500 ts_manager.cc:194] Registered new tserver with Master: 73a70177480547aebc6eddbbde64eefd (127.24.79.194:46423)
I20260808 02:17:54.822059 24895 external_mini_cluster.cc:1428] Setting key badd5b9bf7f61738ad079b8bde4cfef0
I20260808 02:17:54.822698 27500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.194@KRBTEST.COM'} at 127.24.79.194:40087
I20260808 02:17:54.831141 27578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:54.822266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:50537 (local address 127.24.79.254:32993)
0808 02:17:54.822384 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:54.822388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:54.822634 (+ 246us) server_negotiation.cc:407] Connection header received
0808 02:17:54.822775 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:54.822778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:54.822975 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:54.823064 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:54.823788 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:54.824476 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:54.825505 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:54.825811 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:54.825905 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:54.827292 (+ 1387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:54.827295 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:54.827297 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:54.827324 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:54.828995 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:54.829572 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:54.829576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:54.829577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:54.829675 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:54.830017 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:54.830020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:54.830021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:54.830334 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:54.830423 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:54.830841 (+ 418us) server_negotiation.cc:299] Negotiation successful
0808 02:17:54.831004 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 02:17:54.831233 28026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:54.822208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:32993 (local address 127.24.79.194:50537)
0808 02:17:54.822549 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:54.822562 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:54.822675 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:54.823248 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:54.823251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:54.823492 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:54.823628 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:54.823637 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:54.824621 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:54.824625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:54.825380 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:54.825391 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:54.825930 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:54.825934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:54.826030 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:54.826587 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:54.826604 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:54.827133 (+ 529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:54.829145 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:54.829149 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:54.829151 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:54.829457 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:54.829781 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:54.829784 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:54.829787 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:54.829902 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:54.830494 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:54.830498 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:54.830632 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 02:17:54.830778 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 02:17:54.831006 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":3,"thread_start_us":93,"threads_started":1}
WARNING: no policy specified for kudu/127.24.79.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
I20260808 02:17:54.859236 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:32993
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:54.981583 28029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:54.981904 28029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:54.981977 28029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:54.986218 28029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:54.986321 28029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:54.986378 28029 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 02:17:54.986399 28029 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:17:54.986481 28029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:17:54.992285 28029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.79.254:32993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:17:54.993566 28029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:54.994699 28029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:55.006672 28036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:55.006678 28039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:55.006939 28029 server_base.cc:1034] running on GCE node
W20260808 02:17:55.006676 28041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:55.007755 28029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:55.008574 28029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:55.009802 28029 hybrid_clock.cc:648] HybridClock initialized: now 1786155475009772 us; error 42 us; skew 500 ppm
Aug 08 02:17:55 dist-test-slave-1vmg krb5kdc[27437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155475, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:55.013816 28029 init.cc:377] Logged in from keytab as kudu/127.24.79.195@KRBTEST.COM (short username kudu)
I20260808 02:17:55.015040 28029 webserver.cc:460] Webserver started at http://127.24.79.195:40287/ using document root <none> and password file <none>
I20260808 02:17:55.015762 28029 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:55.015841 28029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:55.016063 28029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:55.018383 28029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "477d4996df90406180e96af5e56bdd6e"
format_stamp: "Formatted at 2026-08-08 02:17:55 on dist-test-slave-1vmg"
server_key: "e75148611ec864da337bd9626faadcca"
server_key_iv: "619565fbf6ae9834e4a5ae46da91cae3"
server_key_version: "encryptionkey@0"
I20260808 02:17:55.019140 28029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "477d4996df90406180e96af5e56bdd6e"
format_stamp: "Formatted at 2026-08-08 02:17:55 on dist-test-slave-1vmg"
server_key: "e75148611ec864da337bd9626faadcca"
server_key_iv: "619565fbf6ae9834e4a5ae46da91cae3"
server_key_version: "encryptionkey@0"
I20260808 02:17:55.023970 28029 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:17:55.027026 28053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:55.028654 28029 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:17:55.028820 28029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "477d4996df90406180e96af5e56bdd6e"
format_stamp: "Formatted at 2026-08-08 02:17:55 on dist-test-slave-1vmg"
server_key: "e75148611ec864da337bd9626faadcca"
server_key_iv: "619565fbf6ae9834e4a5ae46da91cae3"
server_key_version: "encryptionkey@0"
I20260808 02:17:55.028954 28029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:55.067988 28029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:55.071777 28029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:55.071995 28029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:55.073876 28029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:17:55.073947 28029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:55.074009 28029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:17:55.074041 28029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:55.085645 28029 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:44031
Aug 08 02:17:55 dist-test-slave-1vmg krb5kdc[27437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155475, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:17:55.085755 28220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:44031 every 8 connection(s)
I20260808 02:17:55.086792 28029 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:17:55.090147 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 28029
I20260808 02:17:55.090265 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:17:55.090516 24895 external_mini_cluster.cc:1428] Setting key cd7b624b34e24ef01951f3484580f6e0
I20260808 02:17:55.091305 27578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.074855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:54281 (local address 127.24.79.254:32993)
0808 02:17:55.075018 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:55.075022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:55.076439 (+ 1417us) server_negotiation.cc:407] Connection header received
0808 02:17:55.077580 (+ 1141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:55.077584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:55.077653 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:55.077744 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:55.079741 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.080585 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.082569 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.082808 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.082893 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.086571 (+ 3678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:55.086575 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:55.086577 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:55.086610 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:55.088459 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:55.089061 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:55.089065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:55.089066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:55.089208 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:55.089546 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:55.089549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:55.089550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:55.089768 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:55.089848 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:55.091027 (+ 1179us) server_negotiation.cc:299] Negotiation successful
0808 02:17:55.091161 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 02:17:55.091776 28068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.075590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:32993 (local address 127.24.79.195:54281)
0808 02:17:55.076216 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:55.076274 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:55.077319 (+ 1045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:55.078008 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:55.078019 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:55.078623 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:55.079488 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.079515 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.080757 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.080760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.082404 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.082417 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.083018 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.083023 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.083253 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.084353 (+ 1100us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:55.084379 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:55.086326 (+ 1947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:55.088601 (+ 2275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:55.088608 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:55.088624 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:55.088928 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:55.089290 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:55.089293 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:55.089294 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:55.089445 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:55.089847 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:55.089871 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:55.090145 (+ 274us) client_negotiation.cc:769] Sending connection context
0808 02:17:55.090336 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 02:17:55.090680 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":141,"threads_started":1}
I20260808 02:17:55.093071 28221 heartbeater.cc:347] Connected to a master server at 127.24.79.254:32993
I20260808 02:17:55.093245 28221 heartbeater.cc:464] Registering TS with master...
I20260808 02:17:55.093897 28221 heartbeater.cc:511] Master 127.24.79.254:32993 requested a full tablet report, sending...
I20260808 02:17:55.095216 27501 ts_manager.cc:194] Registered new tserver with Master: 477d4996df90406180e96af5e56bdd6e (127.24.79.195:44031)
I20260808 02:17:55.096028 27501 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.195@KRBTEST.COM'} at 127.24.79.195:54281
I20260808 02:17:55.103276 27578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.096120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:58533 (local address 127.24.79.254:32993)
0808 02:17:55.096409 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:55.096414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:55.096501 (+ 87us) server_negotiation.cc:407] Connection header received
0808 02:17:55.096636 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:55.096639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:55.096681 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:55.096746 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:55.097246 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.097747 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.098549 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.098768 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.098858 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.100103 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:55.100107 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:55.100109 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:55.100134 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:55.101540 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:55.101995 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:55.101998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:55.101999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:55.102084 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:55.102358 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:55.102361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:55.102362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:55.102562 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:55.102653 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:55.102982 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 02:17:55.103105 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209}
I20260808 02:17:55.103281 28227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.095991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:32993 (local address 127.24.79.195:58533)
0808 02:17:55.096430 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:55.096445 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:55.096549 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:55.096782 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:55.096785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:55.097004 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:55.097116 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.097122 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.097873 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.097877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.098433 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.098441 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.098876 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.098880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.098998 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.099553 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:55.099566 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:55.099950 (+ 384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:55.101658 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:55.101662 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:55.101664 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:55.101894 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:55.102173 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:55.102176 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:55.102178 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:55.102270 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:55.102656 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:55.102659 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:55.102772 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 02:17:55.102860 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 02:17:55.103071 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":141,"threads_started":1}
I20260808 02:17:55.104873 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 02:17:55 dist-test-slave-1vmg krb5kdc[27437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 02:17:55 dist-test-slave-1vmg krb5kdc[27437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:17:55.136431 27578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.126497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49196 (local address 127.24.79.254:32993)
0808 02:17:55.126653 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:55.126657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:55.126671 (+ 14us) server_negotiation.cc:407] Connection header received
0808 02:17:55.126723 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:55.126726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:55.126773 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:55.126854 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:55.127889 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.128544 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.129609 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.129844 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.129943 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.132747 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:55.132751 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:55.132753 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:55.132791 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:55.134536 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:55.135064 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:55.135068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:55.135069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:55.135170 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:55.135470 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:55.135473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:55.135474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:55.135686 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:55.135814 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:55.136165 (+ 351us) server_negotiation.cc:299] Negotiation successful
0808 02:17:55.136297 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 02:17:55.140120 27499 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:17:55.143613 27499 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:17:55.160588 28255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.154856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45840 (local address 127.24.79.195:44031)
0808 02:17:55.155644 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:55.155656 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:55.155680 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:17:55.155759 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:55.155763 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:55.155942 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:55.156367 (+ 425us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:55.156516 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.157647 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.159090 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.159790 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.159896 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.160090 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:55.160297 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 02:17:55.160416 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":172,"threads_started":1}
I20260808 02:17:55.160573 28245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.153943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:44031 (local address 127.0.0.1:45840)
0808 02:17:55.154808 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:55.154873 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:55.155081 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:55.156113 (+ 1032us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:55.156116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:55.156131 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:55.156221 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.156227 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.157782 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.157786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.158904 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.158954 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.159919 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.159923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.160062 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.160105 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 02:17:55.160232 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 02:17:55.160393 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"mutex_wait_us":20,"spinlock_wait_cycles":31232,"thread_start_us":148,"threads_started":1}
I20260808 02:17:55.160931 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.154004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59862 (local address 127.24.79.194:46423)
0808 02:17:55.154495 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:55.154507 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:55.155255 (+ 748us) server_negotiation.cc:407] Connection header received
0808 02:17:55.155355 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:55.155361 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:55.155625 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:55.155803 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:55.156329 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.157607 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.159099 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.159888 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.159975 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.160135 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:55.160678 (+ 543us) server_negotiation.cc:299] Negotiation successful
0808 02:17:55.160794 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":123,"threads_started":1}
I20260808 02:17:55.160948 28254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.154853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54834 (local address 127.24.79.193:45371)
0808 02:17:55.155400 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:55.155407 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:55.155658 (+ 251us) server_negotiation.cc:407] Connection header received
0808 02:17:55.155751 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:55.155756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:55.155964 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:55.156111 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:55.156515 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.157242 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.158943 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.159832 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.159957 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.160119 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:55.160661 (+ 542us) server_negotiation.cc:299] Negotiation successful
0808 02:17:55.160783 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":137,"threads_started":1}
I20260808 02:17:55.160573 28247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.154114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:45371 (local address 127.0.0.1:54834)
0808 02:17:55.155397 (+ 1283us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:55.155421 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:55.155546 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:55.156276 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:55.156279 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:55.156298 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:55.156393 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.156398 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.157372 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.157376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.158766 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.158779 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.160170 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.160173 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.160320 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.160344 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 02:17:55.160407 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 02:17:55.160465 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1209,"mutex_wait_us":19,"thread_start_us":96,"threads_started":1}
I20260808 02:17:55.162015 28249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.153847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:46423 (local address 127.0.0.1:59862)
0808 02:17:55.154808 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:55.154873 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:55.155081 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:55.155999 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:55.156006 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:55.156031 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:55.156192 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.156200 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.157744 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.157748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.158956 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.158966 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.160200 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.160204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.160375 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.160400 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 02:17:55.160469 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 02:17:55.160542 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":774,"mutex_wait_us":71,"spinlock_wait_cycles":3584,"thread_start_us":96,"threads_started":1}
I20260808 02:17:55.164445 27713 tablet_service.cc:1467] Processing CreateTablet for tablet 7f335836e30346ea8c7007a287cfb1ea (DEFAULT_TABLE table=test-table [id=3fc5c7497b1741c4ac7e589180faa5f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:55.164467 28119 tablet_service.cc:1467] Processing CreateTablet for tablet 7f335836e30346ea8c7007a287cfb1ea (DEFAULT_TABLE table=test-table [id=3fc5c7497b1741c4ac7e589180faa5f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:55.165056 27922 tablet_service.cc:1467] Processing CreateTablet for tablet 7f335836e30346ea8c7007a287cfb1ea (DEFAULT_TABLE table=test-table [id=3fc5c7497b1741c4ac7e589180faa5f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:55.165656 27713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f335836e30346ea8c7007a287cfb1ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:55.165673 28119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f335836e30346ea8c7007a287cfb1ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:55.166024 27922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f335836e30346ea8c7007a287cfb1ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:55.172891 28256 tablet_bootstrap.cc:492] T 7f335836e30346ea8c7007a287cfb1ea P 8aed7f68aa194ebba090205fee848e85: Bootstrap starting.
I20260808 02:17:55.174355 28259 tablet_bootstrap.cc:492] T 7f335836e30346ea8c7007a287cfb1ea P 477d4996df90406180e96af5e56bdd6e: Bootstrap starting.
I20260808 02:17:55.175114 28257 tablet_bootstrap.cc:492] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd: Bootstrap starting.
I20260808 02:17:55.176041 28256 tablet_bootstrap.cc:654] T 7f335836e30346ea8c7007a287cfb1ea P 8aed7f68aa194ebba090205fee848e85: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:55.176980 28259 tablet_bootstrap.cc:654] T 7f335836e30346ea8c7007a287cfb1ea P 477d4996df90406180e96af5e56bdd6e: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:55.177102 28257 tablet_bootstrap.cc:654] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:55.177363 28256 log.cc:824] T 7f335836e30346ea8c7007a287cfb1ea P 8aed7f68aa194ebba090205fee848e85: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:55.177983 28257 log.cc:824] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:55.177999 28259 log.cc:824] T 7f335836e30346ea8c7007a287cfb1ea P 477d4996df90406180e96af5e56bdd6e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:55.180321 28257 tablet_bootstrap.cc:492] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd: No bootstrap required, opened a new log
I20260808 02:17:55.180398 28256 tablet_bootstrap.cc:492] T 7f335836e30346ea8c7007a287cfb1ea P 8aed7f68aa194ebba090205fee848e85: No bootstrap required, opened a new log
I20260808 02:17:55.180568 28257 ts_tablet_manager.cc:1397] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 02:17:55.180588 28259 tablet_bootstrap.cc:492] T 7f335836e30346ea8c7007a287cfb1ea P 477d4996df90406180e96af5e56bdd6e: No bootstrap required, opened a new log
I20260808 02:17:55.180724 28256 ts_tablet_manager.cc:1397] T 7f335836e30346ea8c7007a287cfb1ea P 8aed7f68aa194ebba090205fee848e85: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 02:17:55.180744 28259 ts_tablet_manager.cc:1397] T 7f335836e30346ea8c7007a287cfb1ea P 477d4996df90406180e96af5e56bdd6e: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 02:17:55.183974 28257 raft_consensus.cc:348] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } } peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } }
I20260808 02:17:55.184269 28257 raft_consensus.cc:374] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:55.184363 28257 raft_consensus.cc:729] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73a70177480547aebc6eddbbde64eefd, State: Initialized, Role: FOLLOWER
I20260808 02:17:55.184921 28257 consensus_queue.cc:260] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } } peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } }
I20260808 02:17:55.185014 28259 raft_consensus.cc:348] T 7f335836e30346ea8c7007a287cfb1ea P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } } peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } }
I20260808 02:17:55.185361 28259 raft_consensus.cc:374] T 7f335836e30346ea8c7007a287cfb1ea P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:55.185436 28259 raft_consensus.cc:729] T 7f335836e30346ea8c7007a287cfb1ea P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 477d4996df90406180e96af5e56bdd6e, State: Initialized, Role: FOLLOWER
I20260808 02:17:55.185297 28256 raft_consensus.cc:348] T 7f335836e30346ea8c7007a287cfb1ea P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } } peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } }
I20260808 02:17:55.185652 28256 raft_consensus.cc:374] T 7f335836e30346ea8c7007a287cfb1ea P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:55.185734 28256 raft_consensus.cc:729] T 7f335836e30346ea8c7007a287cfb1ea P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aed7f68aa194ebba090205fee848e85, State: Initialized, Role: FOLLOWER
I20260808 02:17:55.185981 28259 consensus_queue.cc:260] T 7f335836e30346ea8c7007a287cfb1ea P 477d4996df90406180e96af5e56bdd6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } } peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } }
I20260808 02:17:55.186244 28257 ts_tablet_manager.cc:1428] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 02:17:55.186254 28256 consensus_queue.cc:260] T 7f335836e30346ea8c7007a287cfb1ea P 8aed7f68aa194ebba090205fee848e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } } peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } }
I20260808 02:17:55.186672 28018 heartbeater.cc:503] Master 127.24.79.254:32993 was elected leader, sending a full tablet report...
I20260808 02:17:55.187130 28259 ts_tablet_manager.cc:1428] T 7f335836e30346ea8c7007a287cfb1ea P 477d4996df90406180e96af5e56bdd6e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 02:17:55.187232 28256 ts_tablet_manager.cc:1428] T 7f335836e30346ea8c7007a287cfb1ea P 8aed7f68aa194ebba090205fee848e85: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 02:17:55.187529 28221 heartbeater.cc:503] Master 127.24.79.254:32993 was elected leader, sending a full tablet report...
I20260808 02:17:55.187747 27809 heartbeater.cc:503] Master 127.24.79.254:32993 was elected leader, sending a full tablet report...
W20260808 02:17:55.306885 27811 tablet.cc:2369] T 7f335836e30346ea8c7007a287cfb1ea P 8aed7f68aa194ebba090205fee848e85: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:17:55.316205 28020 tablet.cc:2369] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:17:55.320518 28267 raft_consensus.cc:482] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:17:55.320796 28267 raft_consensus.cc:504] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } } peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } }
I20260808 02:17:55.322119 28267 leader_election.cc:288] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 477d4996df90406180e96af5e56bdd6e (127.24.79.195:44031), 8aed7f68aa194ebba090205fee848e85 (127.24.79.193:45371)
I20260808 02:17:55.327060 28270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.322485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:44031 (local address 127.24.79.194:39653)
0808 02:17:55.322897 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:55.322942 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:55.323059 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:55.323310 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:55.323313 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:55.323356 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:55.323461 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.323466 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.324361 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.324365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.325489 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.325503 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.326546 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.326550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.326660 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.326686 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 02:17:55.326760 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 02:17:55.326844 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":112,"spinlock_wait_cycles":768,"thread_start_us":104,"threads_started":1}
I20260808 02:17:55.327121 28255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.322636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:39653 (local address 127.24.79.195:44031)
0808 02:17:55.322857 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:55.322862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:55.323016 (+ 154us) server_negotiation.cc:407] Connection header received
0808 02:17:55.323161 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:55.323164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:55.323207 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:55.323285 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:55.323575 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.324226 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.325651 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.326261 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.326361 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.326442 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:55.326811 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 02:17:55.326962 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260808 02:17:55.327227 28254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.322561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:60775 (local address 127.24.79.193:45371)
0808 02:17:55.322808 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:55.322813 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:55.322941 (+ 128us) server_negotiation.cc:407] Connection header received
0808 02:17:55.323122 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:55.323125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:55.323176 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:55.323233 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:55.323577 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.324222 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.325693 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.326412 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.326545 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.326601 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:55.326975 (+ 374us) server_negotiation.cc:299] Negotiation successful
0808 02:17:55.327087 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260808 02:17:55.327739 28271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.322461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:45371 (local address 127.24.79.194:60775)
0808 02:17:55.322786 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:55.322813 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:55.322998 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:55.323271 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:55.323273 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:55.323292 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:55.323453 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.323459 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.324339 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.324343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.325495 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.325503 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.326673 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.326678 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.326786 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.326813 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 02:17:55.327469 (+ 656us) client_negotiation.cc:240] Negotiation successful
0808 02:17:55.327557 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":55,"spinlock_wait_cycles":32256,"thread_start_us":135,"threads_started":1}
I20260808 02:17:55.328619 27742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f335836e30346ea8c7007a287cfb1ea" candidate_uuid: "73a70177480547aebc6eddbbde64eefd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8aed7f68aa194ebba090205fee848e85" is_pre_election: true
I20260808 02:17:55.329046 27742 raft_consensus.cc:2454] T 7f335836e30346ea8c7007a287cfb1ea P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73a70177480547aebc6eddbbde64eefd in term 0.
I20260808 02:17:55.329844 27846 leader_election.cc:302] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73a70177480547aebc6eddbbde64eefd, 8aed7f68aa194ebba090205fee848e85; no voters:
I20260808 02:17:55.329821 28156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f335836e30346ea8c7007a287cfb1ea" candidate_uuid: "73a70177480547aebc6eddbbde64eefd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477d4996df90406180e96af5e56bdd6e" is_pre_election: true
I20260808 02:17:55.330188 28267 raft_consensus.cc:2781] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:17:55.330219 28156 raft_consensus.cc:2454] T 7f335836e30346ea8c7007a287cfb1ea P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73a70177480547aebc6eddbbde64eefd in term 0.
I20260808 02:17:55.330293 28267 raft_consensus.cc:482] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:17:55.330340 28267 raft_consensus.cc:3037] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:55.332384 28267 raft_consensus.cc:504] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } } peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } }
I20260808 02:17:55.332877 28267 leader_election.cc:288] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd [CANDIDATE]: Term 1 election: Requested vote from peers 477d4996df90406180e96af5e56bdd6e (127.24.79.195:44031), 8aed7f68aa194ebba090205fee848e85 (127.24.79.193:45371)
I20260808 02:17:55.333343 28156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f335836e30346ea8c7007a287cfb1ea" candidate_uuid: "73a70177480547aebc6eddbbde64eefd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477d4996df90406180e96af5e56bdd6e"
I20260808 02:17:55.333344 27742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f335836e30346ea8c7007a287cfb1ea" candidate_uuid: "73a70177480547aebc6eddbbde64eefd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8aed7f68aa194ebba090205fee848e85"
I20260808 02:17:55.333480 28156 raft_consensus.cc:3037] T 7f335836e30346ea8c7007a287cfb1ea P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:55.333495 27742 raft_consensus.cc:3037] T 7f335836e30346ea8c7007a287cfb1ea P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:55.334712 28156 raft_consensus.cc:2454] T 7f335836e30346ea8c7007a287cfb1ea P 477d4996df90406180e96af5e56bdd6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73a70177480547aebc6eddbbde64eefd in term 1.
I20260808 02:17:55.334723 27742 raft_consensus.cc:2454] T 7f335836e30346ea8c7007a287cfb1ea P 8aed7f68aa194ebba090205fee848e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73a70177480547aebc6eddbbde64eefd in term 1.
I20260808 02:17:55.335115 27842 leader_election.cc:302] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 477d4996df90406180e96af5e56bdd6e, 73a70177480547aebc6eddbbde64eefd; no voters:
I20260808 02:17:55.335328 28267 raft_consensus.cc:2781] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:55.335529 28267 raft_consensus.cc:686] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd [term 1 LEADER]: Becoming Leader. State: Replica: 73a70177480547aebc6eddbbde64eefd, State: Running, Role: LEADER
I20260808 02:17:55.335892 28267 consensus_queue.cc:237] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } } peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } }
W20260808 02:17:55.338141 28222 tablet.cc:2369] T 7f335836e30346ea8c7007a287cfb1ea P 477d4996df90406180e96af5e56bdd6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:17:55.339484 27501 catalog_manager.cc:5495] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd reported cstate change: term changed from 0 to 1, leader changed from <none> to 73a70177480547aebc6eddbbde64eefd (127.24.79.194). New cstate: current_term: 1 leader_uuid: "73a70177480547aebc6eddbbde64eefd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } health_report { overall_health: UNKNOWN } } }
Aug 08 02:17:55 dist-test-slave-1vmg krb5kdc[27437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155474, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:17:55.373783 28282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.365039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:32993 (local address 127.0.0.1:49204)
0808 02:17:55.365359 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:55.365372 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:55.365499 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:55.365821 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:55.365824 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:55.366277 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:55.366383 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.366390 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.367185 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.367188 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.367755 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.367764 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.368198 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.368201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.368278 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.368853 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:55.368865 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:55.370460 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:55.372158 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:55.372171 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:55.372179 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:55.372420 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:55.372685 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:55.372688 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:55.372691 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:55.372771 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:55.373161 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:55.373169 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:55.373295 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 02:17:55.373396 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 02:17:55.373581 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":128,"threads_started":1}
I20260808 02:17:55.373781 27578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.365097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49204 (local address 127.24.79.254:32993)
0808 02:17:55.365249 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:55.365254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:55.365451 (+ 197us) server_negotiation.cc:407] Connection header received
0808 02:17:55.365667 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:55.365669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:55.365713 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:55.365777 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:55.366504 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.367085 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.367873 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.368093 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.368181 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.370627 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:55.370630 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:55.370632 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:55.370656 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:55.372052 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:55.372515 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:55.372518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:55.372519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:55.372603 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:55.372851 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:55.372855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:55.372856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:55.373058 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:55.373148 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:55.373463 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 02:17:55.373655 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":78,"server-negotiator.queue_time_us":59}
I20260808 02:17:55.378499 27501 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:49204:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 02:17:55.379478 27501 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:17:55.385149 27713 tablet_service.cc:1467] Processing CreateTablet for tablet 320a3f47b87a40258f4f1fa75f88f6a2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d48913ff625e4909bad4a61862738418]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:17:55.385138 27922 tablet_service.cc:1467] Processing CreateTablet for tablet 320a3f47b87a40258f4f1fa75f88f6a2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d48913ff625e4909bad4a61862738418]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:17:55.385205 28119 tablet_service.cc:1467] Processing CreateTablet for tablet 320a3f47b87a40258f4f1fa75f88f6a2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d48913ff625e4909bad4a61862738418]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:17:55.385912 27713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 320a3f47b87a40258f4f1fa75f88f6a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:55.385914 28119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 320a3f47b87a40258f4f1fa75f88f6a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:55.385914 27922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 320a3f47b87a40258f4f1fa75f88f6a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:55.388906 28256 tablet_bootstrap.cc:492] T 320a3f47b87a40258f4f1fa75f88f6a2 P 8aed7f68aa194ebba090205fee848e85: Bootstrap starting.
I20260808 02:17:55.389084 28259 tablet_bootstrap.cc:492] T 320a3f47b87a40258f4f1fa75f88f6a2 P 477d4996df90406180e96af5e56bdd6e: Bootstrap starting.
I20260808 02:17:55.389487 28257 tablet_bootstrap.cc:492] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd: Bootstrap starting.
I20260808 02:17:55.389981 28256 tablet_bootstrap.cc:654] T 320a3f47b87a40258f4f1fa75f88f6a2 P 8aed7f68aa194ebba090205fee848e85: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:55.390134 28259 tablet_bootstrap.cc:654] T 320a3f47b87a40258f4f1fa75f88f6a2 P 477d4996df90406180e96af5e56bdd6e: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:55.390493 28257 tablet_bootstrap.cc:654] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:55.391713 28256 tablet_bootstrap.cc:492] T 320a3f47b87a40258f4f1fa75f88f6a2 P 8aed7f68aa194ebba090205fee848e85: No bootstrap required, opened a new log
I20260808 02:17:55.391736 28259 tablet_bootstrap.cc:492] T 320a3f47b87a40258f4f1fa75f88f6a2 P 477d4996df90406180e96af5e56bdd6e: No bootstrap required, opened a new log
I20260808 02:17:55.391808 28259 ts_tablet_manager.cc:1397] T 320a3f47b87a40258f4f1fa75f88f6a2 P 477d4996df90406180e96af5e56bdd6e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:17:55.391808 28256 ts_tablet_manager.cc:1397] T 320a3f47b87a40258f4f1fa75f88f6a2 P 8aed7f68aa194ebba090205fee848e85: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:17:55.391995 28257 tablet_bootstrap.cc:492] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd: No bootstrap required, opened a new log
I20260808 02:17:55.392078 28257 ts_tablet_manager.cc:1397] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:17:55.392287 28256 raft_consensus.cc:348] T 320a3f47b87a40258f4f1fa75f88f6a2 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:55.392287 28259 raft_consensus.cc:348] T 320a3f47b87a40258f4f1fa75f88f6a2 P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:55.392400 28259 raft_consensus.cc:374] T 320a3f47b87a40258f4f1fa75f88f6a2 P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:55.392400 28256 raft_consensus.cc:374] T 320a3f47b87a40258f4f1fa75f88f6a2 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:55.392431 28259 raft_consensus.cc:729] T 320a3f47b87a40258f4f1fa75f88f6a2 P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 477d4996df90406180e96af5e56bdd6e, State: Initialized, Role: FOLLOWER
I20260808 02:17:55.392431 28256 raft_consensus.cc:729] T 320a3f47b87a40258f4f1fa75f88f6a2 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aed7f68aa194ebba090205fee848e85, State: Initialized, Role: FOLLOWER
I20260808 02:17:55.392441 28257 raft_consensus.cc:348] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:55.392529 28257 raft_consensus.cc:374] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:55.392561 28257 raft_consensus.cc:729] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73a70177480547aebc6eddbbde64eefd, State: Initialized, Role: FOLLOWER
I20260808 02:17:55.392565 28259 consensus_queue.cc:260] T 320a3f47b87a40258f4f1fa75f88f6a2 P 477d4996df90406180e96af5e56bdd6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:55.392565 28256 consensus_queue.cc:260] T 320a3f47b87a40258f4f1fa75f88f6a2 P 8aed7f68aa194ebba090205fee848e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:55.392704 28257 consensus_queue.cc:260] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:55.392830 28259 ts_tablet_manager.cc:1428] T 320a3f47b87a40258f4f1fa75f88f6a2 P 477d4996df90406180e96af5e56bdd6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:17:55.392832 28256 ts_tablet_manager.cc:1428] T 320a3f47b87a40258f4f1fa75f88f6a2 P 8aed7f68aa194ebba090205fee848e85: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:17:55.392982 28267 tablet_replica.cc:439] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } } }
I20260808 02:17:55.393127 28267 tablet_replica.cc:442] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:17:55.393061 28269 tablet_replica.cc:439] T 320a3f47b87a40258f4f1fa75f88f6a2 P 8aed7f68aa194ebba090205fee848e85: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } } }
I20260808 02:17:55.393080 28268 tablet_replica.cc:439] T 320a3f47b87a40258f4f1fa75f88f6a2 P 477d4996df90406180e96af5e56bdd6e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } } }
I20260808 02:17:55.393201 28269 tablet_replica.cc:442] T 320a3f47b87a40258f4f1fa75f88f6a2 P 8aed7f68aa194ebba090205fee848e85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:17:55.393201 28268 tablet_replica.cc:442] T 320a3f47b87a40258f4f1fa75f88f6a2 P 477d4996df90406180e96af5e56bdd6e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:17:55.392982 28257 ts_tablet_manager.cc:1428] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:17:55.414110 28267 raft_consensus.cc:482] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:17:55.414276 28267 raft_consensus.cc:504] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:55.414733 28267 leader_election.cc:288] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8aed7f68aa194ebba090205fee848e85 (127.24.79.193:45371), 477d4996df90406180e96af5e56bdd6e (127.24.79.195:44031)
I20260808 02:17:55.415372 28156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "320a3f47b87a40258f4f1fa75f88f6a2" candidate_uuid: "73a70177480547aebc6eddbbde64eefd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477d4996df90406180e96af5e56bdd6e" is_pre_election: true
I20260808 02:17:55.415372 27742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "320a3f47b87a40258f4f1fa75f88f6a2" candidate_uuid: "73a70177480547aebc6eddbbde64eefd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8aed7f68aa194ebba090205fee848e85" is_pre_election: true
I20260808 02:17:55.415555 28156 raft_consensus.cc:2454] T 320a3f47b87a40258f4f1fa75f88f6a2 P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73a70177480547aebc6eddbbde64eefd in term 0.
I20260808 02:17:55.415556 27742 raft_consensus.cc:2454] T 320a3f47b87a40258f4f1fa75f88f6a2 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73a70177480547aebc6eddbbde64eefd in term 0.
I20260808 02:17:55.415899 27842 leader_election.cc:302] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 477d4996df90406180e96af5e56bdd6e, 73a70177480547aebc6eddbbde64eefd, 8aed7f68aa194ebba090205fee848e85; no voters:
I20260808 02:17:55.416157 28267 raft_consensus.cc:2781] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:17:55.416276 28267 raft_consensus.cc:482] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:17:55.416321 28267 raft_consensus.cc:3037] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:55.417383 28267 raft_consensus.cc:504] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:55.417749 28267 leader_election.cc:288] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd [CANDIDATE]: Term 1 election: Requested vote from peers 8aed7f68aa194ebba090205fee848e85 (127.24.79.193:45371), 477d4996df90406180e96af5e56bdd6e (127.24.79.195:44031)
I20260808 02:17:55.418200 27742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "320a3f47b87a40258f4f1fa75f88f6a2" candidate_uuid: "73a70177480547aebc6eddbbde64eefd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8aed7f68aa194ebba090205fee848e85"
I20260808 02:17:55.418329 27742 raft_consensus.cc:3037] T 320a3f47b87a40258f4f1fa75f88f6a2 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:55.419140 28156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "320a3f47b87a40258f4f1fa75f88f6a2" candidate_uuid: "73a70177480547aebc6eddbbde64eefd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477d4996df90406180e96af5e56bdd6e"
I20260808 02:17:55.419339 28156 raft_consensus.cc:3037] T 320a3f47b87a40258f4f1fa75f88f6a2 P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:55.419479 27742 raft_consensus.cc:2454] T 320a3f47b87a40258f4f1fa75f88f6a2 P 8aed7f68aa194ebba090205fee848e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73a70177480547aebc6eddbbde64eefd in term 1.
I20260808 02:17:55.419874 27846 leader_election.cc:302] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73a70177480547aebc6eddbbde64eefd, 8aed7f68aa194ebba090205fee848e85; no voters:
I20260808 02:17:55.420092 28267 raft_consensus.cc:2781] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:55.420199 28267 raft_consensus.cc:686] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd [term 1 LEADER]: Becoming Leader. State: Replica: 73a70177480547aebc6eddbbde64eefd, State: Running, Role: LEADER
I20260808 02:17:55.420356 28267 consensus_queue.cc:237] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:55.420614 28156 raft_consensus.cc:2454] T 320a3f47b87a40258f4f1fa75f88f6a2 P 477d4996df90406180e96af5e56bdd6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73a70177480547aebc6eddbbde64eefd in term 1.
I20260808 02:17:55.421166 28273 tablet_replica.cc:439] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd: TxnStatusTablet state changed. Reason: New leader 73a70177480547aebc6eddbbde64eefd. Latest consensus state: current_term: 1 leader_uuid: "73a70177480547aebc6eddbbde64eefd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } } }
I20260808 02:17:55.421288 28273 tablet_replica.cc:442] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:17:55.421661 28289 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 02:17:55.421979 28289 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 02:17:55.422312 27499 catalog_manager.cc:5495] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd reported cstate change: term changed from 0 to 1, leader changed from <none> to 73a70177480547aebc6eddbbde64eefd (127.24.79.194). New cstate: current_term: 1 leader_uuid: "73a70177480547aebc6eddbbde64eefd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } health_report { overall_health: UNKNOWN } } }
I20260808 02:17:55.443578 28282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.438871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:46423 (local address 127.0.0.1:59878)
0808 02:17:55.439125 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:55.439148 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:55.439293 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:55.439700 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:55.439703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:55.439711 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:17:55.439850 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.439855 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.440891 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.440894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.441530 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.441539 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.442161 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.442165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.442246 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.442307 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:17:55.443256 (+ 949us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:17:55.443278 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 02:17:55.443349 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 02:17:55.443442 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104}
I20260808 02:17:55.443593 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.439123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59878 (local address 127.24.79.194:46423)
0808 02:17:55.439296 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:55.439301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:55.439321 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:17:55.439439 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:55.439443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:55.439519 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:55.439631 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:55.439963 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.440764 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.441640 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.441980 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.442082 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.442460 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:55.443128 (+ 668us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:55.443261 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:55.443374 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 02:17:55.443449 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 02:17:55.449466 28156 raft_consensus.cc:1261] T 320a3f47b87a40258f4f1fa75f88f6a2 P 477d4996df90406180e96af5e56bdd6e [term 1 FOLLOWER]: Refusing update from remote peer 73a70177480547aebc6eddbbde64eefd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:55.449441 27742 raft_consensus.cc:1261] T 320a3f47b87a40258f4f1fa75f88f6a2 P 8aed7f68aa194ebba090205fee848e85 [term 1 FOLLOWER]: Refusing update from remote peer 73a70177480547aebc6eddbbde64eefd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:55.450300 28267 consensus_queue.cc:1035] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd [LEADER]: Connected to new peer: Peer: permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:55.450517 28273 consensus_queue.cc:1035] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:55.455320 28268 tablet_replica.cc:439] T 320a3f47b87a40258f4f1fa75f88f6a2 P 477d4996df90406180e96af5e56bdd6e: TxnStatusTablet state changed. Reason: New leader 73a70177480547aebc6eddbbde64eefd. Latest consensus state: current_term: 1 leader_uuid: "73a70177480547aebc6eddbbde64eefd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } } }
I20260808 02:17:55.455490 28268 tablet_replica.cc:442] T 320a3f47b87a40258f4f1fa75f88f6a2 P 477d4996df90406180e96af5e56bdd6e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:17:55.457412 28269 tablet_replica.cc:439] T 320a3f47b87a40258f4f1fa75f88f6a2 P 8aed7f68aa194ebba090205fee848e85: TxnStatusTablet state changed. Reason: New leader 73a70177480547aebc6eddbbde64eefd. Latest consensus state: current_term: 1 leader_uuid: "73a70177480547aebc6eddbbde64eefd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } } }
I20260808 02:17:55.457592 28269 tablet_replica.cc:442] T 320a3f47b87a40258f4f1fa75f88f6a2 P 8aed7f68aa194ebba090205fee848e85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:17:55.461237 28294 mvcc.cc:205] Tried to move back new op lower bound from 7316092827433115648 to 7316092827323334656. Current Snapshot: MvccSnapshot[applied={T|T < 7316092827433115648}]
I20260808 02:17:55.461889 28268 tablet_replica.cc:439] T 320a3f47b87a40258f4f1fa75f88f6a2 P 477d4996df90406180e96af5e56bdd6e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "73a70177480547aebc6eddbbde64eefd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } } }
I20260808 02:17:55.462046 28268 tablet_replica.cc:442] T 320a3f47b87a40258f4f1fa75f88f6a2 P 477d4996df90406180e96af5e56bdd6e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:17:55.462527 28273 tablet_replica.cc:439] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "73a70177480547aebc6eddbbde64eefd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } } }
I20260808 02:17:55.462656 28273 tablet_replica.cc:442] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:17:55.462648 28302 tablet_replica.cc:439] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "73a70177480547aebc6eddbbde64eefd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } } }
I20260808 02:17:55.463193 28302 tablet_replica.cc:442] T 320a3f47b87a40258f4f1fa75f88f6a2 P 73a70177480547aebc6eddbbde64eefd: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:17:55.463574 28269 tablet_replica.cc:439] T 320a3f47b87a40258f4f1fa75f88f6a2 P 8aed7f68aa194ebba090205fee848e85: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "73a70177480547aebc6eddbbde64eefd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } } }
I20260808 02:17:55.463688 28269 tablet_replica.cc:442] T 320a3f47b87a40258f4f1fa75f88f6a2 P 8aed7f68aa194ebba090205fee848e85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:17:55.465266 28297 mvcc.cc:205] Tried to move back new op lower bound from 7316092827433115648 to 7316092827323334656. Current Snapshot: MvccSnapshot[applied={T|T < 7316092827433115648}]
I20260808 02:17:55.466265 28295 mvcc.cc:205] Tried to move back new op lower bound from 7316092827433115648 to 7316092827323334656. Current Snapshot: MvccSnapshot[applied={T|T < 7316092827433115648}]
I20260808 02:17:55.475354 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.471736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59892 (local address 127.24.79.194:46423)
0808 02:17:55.471866 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:55.471870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:55.471884 (+ 14us) server_negotiation.cc:407] Connection header received
0808 02:17:55.472000 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:55.472003 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:55.472047 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:55.472118 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:55.472378 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.472898 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.474148 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.474384 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.474536 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.474822 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:55.474935 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:55.475001 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:55.475105 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 02:17:55.475163 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 02:17:55.489681 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.485618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:46065 (local address 127.24.79.194:46423)
0808 02:17:55.485768 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:55.485773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:55.486032 (+ 259us) server_negotiation.cc:407] Connection header received
0808 02:17:55.486176 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:55.486179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:55.486230 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:55.486309 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:55.486654 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.487209 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.488456 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.488687 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.488779 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.489209 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:55.489296 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:55.489355 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:55.489480 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 02:17:55.489531 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":56}
I20260808 02:17:55.489725 28308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.485565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:46423 (local address 127.24.79.194:46065)
0808 02:17:55.485924 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:55.485939 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:55.486069 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:55.486336 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:55.486340 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:55.486347 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:17:55.486512 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.486519 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.487310 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.487314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.488305 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.488317 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.488828 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.488832 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.488935 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.489005 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:17:55.489376 (+ 371us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:17:55.489382 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 02:17:55.489446 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 02:17:55.489540 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":20,"thread_start_us":142,"threads_started":1}
I20260808 02:17:55.498134 27742 raft_consensus.cc:1261] T 7f335836e30346ea8c7007a287cfb1ea P 8aed7f68aa194ebba090205fee848e85 [term 1 FOLLOWER]: Refusing update from remote peer 73a70177480547aebc6eddbbde64eefd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:55.498594 28156 raft_consensus.cc:1261] T 7f335836e30346ea8c7007a287cfb1ea P 477d4996df90406180e96af5e56bdd6e [term 1 FOLLOWER]: Refusing update from remote peer 73a70177480547aebc6eddbbde64eefd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:55.498746 28273 consensus_queue.cc:1035] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:55.499038 28302 consensus_queue.cc:1035] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd [LEADER]: Connected to new peer: Peer: permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:55.546780 27499 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:17:55.547080 27499 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:17:55.550213 27499 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1: Sending DeleteTablet for 3 replicas of tablet 7f335836e30346ea8c7007a287cfb1ea
I20260808 02:17:55.551244 28119 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f335836e30346ea8c7007a287cfb1ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:55 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:45840
I20260808 02:17:55.551309 27713 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f335836e30346ea8c7007a287cfb1ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:55 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:54834
I20260808 02:17:55.552194 28313 tablet_replica.cc:330] T 7f335836e30346ea8c7007a287cfb1ea P 477d4996df90406180e96af5e56bdd6e: stopping tablet replica
I20260808 02:17:55.552121 27499 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 02:17:55.552187 27922 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f335836e30346ea8c7007a287cfb1ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:55 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:59862
I20260808 02:17:55.552596 28313 raft_consensus.cc:2229] T 7f335836e30346ea8c7007a287cfb1ea P 477d4996df90406180e96af5e56bdd6e [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 02:17:55.552641 27499 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:17:55.552876 28313 raft_consensus.cc:2258] T 7f335836e30346ea8c7007a287cfb1ea P 477d4996df90406180e96af5e56bdd6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:55.553165 28315 tablet_replica.cc:330] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd: stopping tablet replica
I20260808 02:17:55.553514 28315 raft_consensus.cc:2229] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:17:55.554028 28315 raft_consensus.cc:2258] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:55.554461 28314 tablet_replica.cc:330] T 7f335836e30346ea8c7007a287cfb1ea P 8aed7f68aa194ebba090205fee848e85: stopping tablet replica
I20260808 02:17:55.554848 28314 raft_consensus.cc:2229] T 7f335836e30346ea8c7007a287cfb1ea P 8aed7f68aa194ebba090205fee848e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:17:55.555173 28314 raft_consensus.cc:2258] T 7f335836e30346ea8c7007a287cfb1ea P 8aed7f68aa194ebba090205fee848e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:55.556535 28313 ts_tablet_manager.cc:1905] T 7f335836e30346ea8c7007a287cfb1ea P 477d4996df90406180e96af5e56bdd6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:17:55.557982 28315 ts_tablet_manager.cc:1905] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:17:55.560269 28313 ts_tablet_manager.cc:1918] T 7f335836e30346ea8c7007a287cfb1ea P 477d4996df90406180e96af5e56bdd6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:17:55.560508 28313 log.cc:1196] T 7f335836e30346ea8c7007a287cfb1ea P 477d4996df90406180e96af5e56bdd6e: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/7f335836e30346ea8c7007a287cfb1ea
I20260808 02:17:55.560674 27922 tablet_service.cc:1467] Processing CreateTablet for tablet 6a3819b61fc74f67bc584663779a24d2 (DEFAULT_TABLE table=test-table [id=eadd210fd8d04717a13a75a859df4330]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:55.560755 28119 tablet_service.cc:1467] Processing CreateTablet for tablet 6a3819b61fc74f67bc584663779a24d2 (DEFAULT_TABLE table=test-table [id=eadd210fd8d04717a13a75a859df4330]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:55.560987 27922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a3819b61fc74f67bc584663779a24d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:55.561038 28119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a3819b61fc74f67bc584663779a24d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:55.561633 28313 ts_tablet_manager.cc:1939] T 7f335836e30346ea8c7007a287cfb1ea P 477d4996df90406180e96af5e56bdd6e: Deleting consensus metadata
I20260808 02:17:55.561642 28315 ts_tablet_manager.cc:1918] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:17:55.561748 28315 log.cc:1196] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/7f335836e30346ea8c7007a287cfb1ea
I20260808 02:17:55.563364 28314 ts_tablet_manager.cc:1905] T 7f335836e30346ea8c7007a287cfb1ea P 8aed7f68aa194ebba090205fee848e85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:17:55.563486 27478 catalog_manager.cc:4841] TS 477d4996df90406180e96af5e56bdd6e (127.24.79.195:44031): tablet 7f335836e30346ea8c7007a287cfb1ea (table test-table [id=3fc5c7497b1741c4ac7e589180faa5f4]) successfully deleted
I20260808 02:17:55.563597 28259 tablet_bootstrap.cc:492] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e: Bootstrap starting.
I20260808 02:17:55.564611 28259 tablet_bootstrap.cc:654] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:55.564580 27713 tablet_service.cc:1467] Processing CreateTablet for tablet 6a3819b61fc74f67bc584663779a24d2 (DEFAULT_TABLE table=test-table [id=eadd210fd8d04717a13a75a859df4330]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:55.564888 27713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a3819b61fc74f67bc584663779a24d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:55.566109 28259 tablet_bootstrap.cc:492] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e: No bootstrap required, opened a new log
I20260808 02:17:55.566219 28259 ts_tablet_manager.cc:1397] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:17:55.566709 28259 raft_consensus.cc:348] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:55.566854 28259 raft_consensus.cc:374] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:55.566890 28259 raft_consensus.cc:729] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 477d4996df90406180e96af5e56bdd6e, State: Initialized, Role: FOLLOWER
I20260808 02:17:55.567058 28259 consensus_queue.cc:260] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:55.567364 28259 ts_tablet_manager.cc:1428] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:17:55.567606 28256 tablet_bootstrap.cc:492] T 6a3819b61fc74f67bc584663779a24d2 P 8aed7f68aa194ebba090205fee848e85: Bootstrap starting.
I20260808 02:17:55.568639 28256 tablet_bootstrap.cc:654] T 6a3819b61fc74f67bc584663779a24d2 P 8aed7f68aa194ebba090205fee848e85: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:55.572273 28314 ts_tablet_manager.cc:1918] T 7f335836e30346ea8c7007a287cfb1ea P 8aed7f68aa194ebba090205fee848e85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:17:55.572423 28314 log.cc:1196] T 7f335836e30346ea8c7007a287cfb1ea P 8aed7f68aa194ebba090205fee848e85: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-0/wal/wals/7f335836e30346ea8c7007a287cfb1ea
I20260808 02:17:55.572783 28314 ts_tablet_manager.cc:1939] T 7f335836e30346ea8c7007a287cfb1ea P 8aed7f68aa194ebba090205fee848e85: Deleting consensus metadata
I20260808 02:17:55.563493 28257 tablet_bootstrap.cc:492] T 6a3819b61fc74f67bc584663779a24d2 P 73a70177480547aebc6eddbbde64eefd: Bootstrap starting.
I20260808 02:17:55.574321 27484 catalog_manager.cc:4841] TS 8aed7f68aa194ebba090205fee848e85 (127.24.79.193:45371): tablet 7f335836e30346ea8c7007a287cfb1ea (table test-table [id=3fc5c7497b1741c4ac7e589180faa5f4]) successfully deleted
I20260808 02:17:55.575992 28257 tablet_bootstrap.cc:654] T 6a3819b61fc74f67bc584663779a24d2 P 73a70177480547aebc6eddbbde64eefd: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:55.562103 28315 ts_tablet_manager.cc:1939] T 7f335836e30346ea8c7007a287cfb1ea P 73a70177480547aebc6eddbbde64eefd: Deleting consensus metadata
I20260808 02:17:55.578150 27482 catalog_manager.cc:4841] TS 73a70177480547aebc6eddbbde64eefd (127.24.79.194:46423): tablet 7f335836e30346ea8c7007a287cfb1ea (table test-table [id=3fc5c7497b1741c4ac7e589180faa5f4]) successfully deleted
I20260808 02:17:55.579141 28256 tablet_bootstrap.cc:492] T 6a3819b61fc74f67bc584663779a24d2 P 8aed7f68aa194ebba090205fee848e85: No bootstrap required, opened a new log
I20260808 02:17:55.579280 28256 ts_tablet_manager.cc:1397] T 6a3819b61fc74f67bc584663779a24d2 P 8aed7f68aa194ebba090205fee848e85: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260808 02:17:55.579872 28256 raft_consensus.cc:348] T 6a3819b61fc74f67bc584663779a24d2 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:55.580024 28256 raft_consensus.cc:374] T 6a3819b61fc74f67bc584663779a24d2 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:55.580070 28256 raft_consensus.cc:729] T 6a3819b61fc74f67bc584663779a24d2 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aed7f68aa194ebba090205fee848e85, State: Initialized, Role: FOLLOWER
I20260808 02:17:55.580230 28256 consensus_queue.cc:260] T 6a3819b61fc74f67bc584663779a24d2 P 8aed7f68aa194ebba090205fee848e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:55.580565 28256 ts_tablet_manager.cc:1428] T 6a3819b61fc74f67bc584663779a24d2 P 8aed7f68aa194ebba090205fee848e85: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:17:55.580899 28257 tablet_bootstrap.cc:492] T 6a3819b61fc74f67bc584663779a24d2 P 73a70177480547aebc6eddbbde64eefd: No bootstrap required, opened a new log
I20260808 02:17:55.581000 28257 ts_tablet_manager.cc:1397] T 6a3819b61fc74f67bc584663779a24d2 P 73a70177480547aebc6eddbbde64eefd: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.002s
I20260808 02:17:55.581470 28257 raft_consensus.cc:348] T 6a3819b61fc74f67bc584663779a24d2 P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:55.581616 28257 raft_consensus.cc:374] T 6a3819b61fc74f67bc584663779a24d2 P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:55.581650 28257 raft_consensus.cc:729] T 6a3819b61fc74f67bc584663779a24d2 P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73a70177480547aebc6eddbbde64eefd, State: Initialized, Role: FOLLOWER
I20260808 02:17:55.581792 28257 consensus_queue.cc:260] T 6a3819b61fc74f67bc584663779a24d2 P 73a70177480547aebc6eddbbde64eefd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:55.582073 28257 ts_tablet_manager.cc:1428] T 6a3819b61fc74f67bc584663779a24d2 P 73a70177480547aebc6eddbbde64eefd: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 02:17:55.642745 28268 raft_consensus.cc:482] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:17:55.642966 28268 raft_consensus.cc:504] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:55.644413 28268 leader_election.cc:288] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8aed7f68aa194ebba090205fee848e85 (127.24.79.193:45371), 73a70177480547aebc6eddbbde64eefd (127.24.79.194:46423)
I20260808 02:17:55.648947 28320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.644701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:45371 (local address 127.24.79.195:56323)
0808 02:17:55.645138 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:55.645154 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:55.645335 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:55.645690 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:55.645692 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:55.645707 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:55.645841 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.645847 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.646843 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.646846 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.647738 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.647750 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.648476 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.648480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.648585 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.648614 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 02:17:55.648688 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 02:17:55.648776 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":94,"threads_started":1}
I20260808 02:17:55.648973 28254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.644917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:56323 (local address 127.24.79.193:45371)
0808 02:17:55.645050 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:55.645054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:55.645243 (+ 189us) server_negotiation.cc:407] Connection header received
0808 02:17:55.645457 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:55.645461 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:55.645509 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:55.645554 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:55.645979 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.646714 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.647867 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.648353 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.648479 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.648539 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:55.648724 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 02:17:55.648811 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:17:55.649503 28321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.644701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:46423 (local address 127.24.79.195:51817)
0808 02:17:55.645247 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:55.645259 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:55.645364 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:55.645598 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:55.645600 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:55.645616 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:55.645755 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.645761 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.646817 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.646821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.648092 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.648104 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.649089 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.649093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.649187 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.649210 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:17:55.649282 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 02:17:55.649354 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":338,"thread_start_us":118,"threads_started":1}
I20260808 02:17:55.649525 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.644840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:51817 (local address 127.24.79.194:46423)
0808 02:17:55.644975 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:55.644980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:55.645347 (+ 367us) server_negotiation.cc:407] Connection header received
0808 02:17:55.645460 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:55.645463 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:55.645509 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:55.645574 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:55.645883 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.646691 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.648229 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.648853 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.648967 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.649018 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:55.649306 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 02:17:55.649409 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:17:55.650013 27742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a3819b61fc74f67bc584663779a24d2" candidate_uuid: "477d4996df90406180e96af5e56bdd6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8aed7f68aa194ebba090205fee848e85" is_pre_election: true
I20260808 02:17:55.650094 27952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a3819b61fc74f67bc584663779a24d2" candidate_uuid: "477d4996df90406180e96af5e56bdd6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73a70177480547aebc6eddbbde64eefd" is_pre_election: true
I20260808 02:17:55.650225 27742 raft_consensus.cc:2454] T 6a3819b61fc74f67bc584663779a24d2 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 477d4996df90406180e96af5e56bdd6e in term 0.
I20260808 02:17:55.650341 27952 raft_consensus.cc:2454] T 6a3819b61fc74f67bc584663779a24d2 P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 477d4996df90406180e96af5e56bdd6e in term 0.
I20260808 02:17:55.650655 28061 leader_election.cc:302] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 477d4996df90406180e96af5e56bdd6e, 8aed7f68aa194ebba090205fee848e85; no voters:
I20260808 02:17:55.651012 28268 raft_consensus.cc:2781] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:17:55.651084 28268 raft_consensus.cc:482] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:17:55.651120 28268 raft_consensus.cc:3037] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:55.652377 28268 raft_consensus.cc:504] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:55.652812 28268 leader_election.cc:288] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e [CANDIDATE]: Term 1 election: Requested vote from peers 8aed7f68aa194ebba090205fee848e85 (127.24.79.193:45371), 73a70177480547aebc6eddbbde64eefd (127.24.79.194:46423)
I20260808 02:17:55.653209 27742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a3819b61fc74f67bc584663779a24d2" candidate_uuid: "477d4996df90406180e96af5e56bdd6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8aed7f68aa194ebba090205fee848e85"
I20260808 02:17:55.653209 27952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a3819b61fc74f67bc584663779a24d2" candidate_uuid: "477d4996df90406180e96af5e56bdd6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73a70177480547aebc6eddbbde64eefd"
I20260808 02:17:55.653347 27952 raft_consensus.cc:3037] T 6a3819b61fc74f67bc584663779a24d2 P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:55.653347 27742 raft_consensus.cc:3037] T 6a3819b61fc74f67bc584663779a24d2 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:55.654541 27952 raft_consensus.cc:2454] T 6a3819b61fc74f67bc584663779a24d2 P 73a70177480547aebc6eddbbde64eefd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 477d4996df90406180e96af5e56bdd6e in term 1.
I20260808 02:17:55.654551 27742 raft_consensus.cc:2454] T 6a3819b61fc74f67bc584663779a24d2 P 8aed7f68aa194ebba090205fee848e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 477d4996df90406180e96af5e56bdd6e in term 1.
I20260808 02:17:55.654879 28059 leader_election.cc:302] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 477d4996df90406180e96af5e56bdd6e, 73a70177480547aebc6eddbbde64eefd; no voters:
I20260808 02:17:55.655118 28268 raft_consensus.cc:2781] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:55.655303 28268 raft_consensus.cc:686] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e [term 1 LEADER]: Becoming Leader. State: Replica: 477d4996df90406180e96af5e56bdd6e, State: Running, Role: LEADER
I20260808 02:17:55.655601 28268 consensus_queue.cc:237] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:55.658262 27501 catalog_manager.cc:5495] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 477d4996df90406180e96af5e56bdd6e (127.24.79.195). New cstate: current_term: 1 leader_uuid: "477d4996df90406180e96af5e56bdd6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } health_report { overall_health: HEALTHY } } }
I20260808 02:17:55.689783 28255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.685750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45844 (local address 127.24.79.195:44031)
0808 02:17:55.686052 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:55.686056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:55.686066 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:17:55.686145 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:55.686147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:55.686209 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:55.686298 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:55.686611 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.687184 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.688329 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.688533 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.688604 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.688861 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:55.689368 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:55.689469 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:55.689534 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 02:17:55.689585 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227}
I20260808 02:17:55.702907 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.698201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:50035 (local address 127.24.79.194:46423)
0808 02:17:55.698370 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:55.698374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:55.698499 (+ 125us) server_negotiation.cc:407] Connection header received
0808 02:17:55.698646 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:55.698649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:55.698697 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:55.698761 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:55.699684 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.700266 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.701543 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.701787 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.701885 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.702468 (+ 583us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:55.702557 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:55.702620 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:55.702727 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 02:17:55.702783 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260808 02:17:55.703019 28328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.698035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:46423 (local address 127.24.79.195:50035)
0808 02:17:55.698403 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:55.698420 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:55.698551 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:55.698792 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:55.698795 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:55.698804 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:17:55.699524 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.699537 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.700390 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.700394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.701374 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.701388 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.701911 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.701914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.702016 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.702087 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:17:55.702639 (+ 552us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:17:55.702646 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 02:17:55.702712 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 02:17:55.702810 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":123,"threads_started":1}
I20260808 02:17:55.710350 27742 raft_consensus.cc:1261] T 6a3819b61fc74f67bc584663779a24d2 P 8aed7f68aa194ebba090205fee848e85 [term 1 FOLLOWER]: Refusing update from remote peer 477d4996df90406180e96af5e56bdd6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:55.710635 27952 raft_consensus.cc:1261] T 6a3819b61fc74f67bc584663779a24d2 P 73a70177480547aebc6eddbbde64eefd [term 1 FOLLOWER]: Refusing update from remote peer 477d4996df90406180e96af5e56bdd6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:55.710903 28322 consensus_queue.cc:1035] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:55.711238 28268 consensus_queue.cc:1035] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:55.733574 28255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.729986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:53571 (local address 127.24.79.195:44031)
0808 02:17:55.730184 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:55.730189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:55.730214 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:17:55.730257 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:55.730260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:55.730308 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:55.730364 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:55.730787 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.731350 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.732410 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.732719 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.732815 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.733101 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:55.733239 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:55.733293 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:55.733407 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 02:17:55.733458 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260808 02:17:55.733670 28308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:55.729897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:44031 (local address 127.24.79.194:53571)
0808 02:17:55.730036 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:55.730053 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:55.730150 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:55.730389 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:55.730391 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:55.730399 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:17:55.730528 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.730535 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.731459 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.731463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.732262 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:55.732274 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:55.732832 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:55.732836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:55.732938 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:55.732978 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:17:55.733312 (+ 334us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:17:55.733320 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 02:17:55.733401 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 02:17:55.733500 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 02:17:55.734874 28297 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 02:17:57.305927 27501 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:17:57.306118 27501 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:17:57.309198 27501 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1: Sending DeleteTablet for 3 replicas of tablet 6a3819b61fc74f67bc584663779a24d2
I20260808 02:17:57.309933 27922 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a3819b61fc74f67bc584663779a24d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:57 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:59862
I20260808 02:17:57.309937 27713 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a3819b61fc74f67bc584663779a24d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:57 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:54834
I20260808 02:17:57.310169 28119 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a3819b61fc74f67bc584663779a24d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:57 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:45840
I20260808 02:17:57.310367 28373 tablet_replica.cc:330] T 6a3819b61fc74f67bc584663779a24d2 P 73a70177480547aebc6eddbbde64eefd: stopping tablet replica
I20260808 02:17:57.310559 28373 raft_consensus.cc:2229] T 6a3819b61fc74f67bc584663779a24d2 P 73a70177480547aebc6eddbbde64eefd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:17:57.310812 28373 raft_consensus.cc:2258] T 6a3819b61fc74f67bc584663779a24d2 P 73a70177480547aebc6eddbbde64eefd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:57.311196 28372 tablet_replica.cc:330] T 6a3819b61fc74f67bc584663779a24d2 P 8aed7f68aa194ebba090205fee848e85: stopping tablet replica
I20260808 02:17:57.311780 28373 ts_tablet_manager.cc:1905] T 6a3819b61fc74f67bc584663779a24d2 P 73a70177480547aebc6eddbbde64eefd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:17:57.312505 28374 tablet_replica.cc:330] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e: stopping tablet replica
I20260808 02:17:57.312597 28372 raft_consensus.cc:2229] T 6a3819b61fc74f67bc584663779a24d2 P 8aed7f68aa194ebba090205fee848e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:17:57.312711 28374 raft_consensus.cc:2229] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:17:57.312891 28372 raft_consensus.cc:2258] T 6a3819b61fc74f67bc584663779a24d2 P 8aed7f68aa194ebba090205fee848e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:57.313174 28374 raft_consensus.cc:2258] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:57.314038 28374 ts_tablet_manager.cc:1905] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:17:57.314414 28372 ts_tablet_manager.cc:1905] T 6a3819b61fc74f67bc584663779a24d2 P 8aed7f68aa194ebba090205fee848e85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:17:57.315095 28373 ts_tablet_manager.cc:1918] T 6a3819b61fc74f67bc584663779a24d2 P 73a70177480547aebc6eddbbde64eefd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:17:57.315182 28373 log.cc:1196] T 6a3819b61fc74f67bc584663779a24d2 P 73a70177480547aebc6eddbbde64eefd: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/6a3819b61fc74f67bc584663779a24d2
I20260808 02:17:57.315546 28373 ts_tablet_manager.cc:1939] T 6a3819b61fc74f67bc584663779a24d2 P 73a70177480547aebc6eddbbde64eefd: Deleting consensus metadata
I20260808 02:17:57.316458 28374 ts_tablet_manager.cc:1918] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:17:57.316525 27482 catalog_manager.cc:4841] TS 73a70177480547aebc6eddbbde64eefd (127.24.79.194:46423): tablet 6a3819b61fc74f67bc584663779a24d2 (table test-table [id=eadd210fd8d04717a13a75a859df4330]) successfully deleted
I20260808 02:17:57.316587 28374 log.cc:1196] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/6a3819b61fc74f67bc584663779a24d2
I20260808 02:17:57.316875 28374 ts_tablet_manager.cc:1939] T 6a3819b61fc74f67bc584663779a24d2 P 477d4996df90406180e96af5e56bdd6e: Deleting consensus metadata
I20260808 02:17:57.317170 28372 ts_tablet_manager.cc:1918] T 6a3819b61fc74f67bc584663779a24d2 P 8aed7f68aa194ebba090205fee848e85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:17:57.317269 28372 log.cc:1196] T 6a3819b61fc74f67bc584663779a24d2 P 8aed7f68aa194ebba090205fee848e85: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-0/wal/wals/6a3819b61fc74f67bc584663779a24d2
I20260808 02:17:57.317551 28372 ts_tablet_manager.cc:1939] T 6a3819b61fc74f67bc584663779a24d2 P 8aed7f68aa194ebba090205fee848e85: Deleting consensus metadata
I20260808 02:17:57.317685 27478 catalog_manager.cc:4841] TS 477d4996df90406180e96af5e56bdd6e (127.24.79.195:44031): tablet 6a3819b61fc74f67bc584663779a24d2 (table test-table [id=eadd210fd8d04717a13a75a859df4330]) successfully deleted
I20260808 02:17:57.318279 27484 catalog_manager.cc:4841] TS 8aed7f68aa194ebba090205fee848e85 (127.24.79.193:45371): tablet 6a3819b61fc74f67bc584663779a24d2 (table test-table [id=eadd210fd8d04717a13a75a859df4330]) successfully deleted
Aug 08 02:17:57 dist-test-slave-1vmg krb5kdc[27437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155477, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:17:57 dist-test-slave-1vmg krb5kdc[27437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155477, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:17:57.346312 28383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:57.336146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49220 (local address 127.24.79.254:32993)
0808 02:17:57.336823 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:57.336830 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:57.336842 (+ 12us) server_negotiation.cc:407] Connection header received
0808 02:17:57.336887 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:57.336889 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:57.336931 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:57.337006 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:57.337912 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:57.338484 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:57.339551 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:57.339885 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:57.340000 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:57.342749 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:57.342753 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:57.342756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:57.342795 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:57.344448 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:57.344948 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:57.344952 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:57.344953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:57.345056 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:57.345417 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:57.345420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:57.345421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:57.345637 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:57.345748 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:57.346004 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 02:17:57.346128 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":114,"threads_started":1}
I20260808 02:17:57.349442 27501 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:17:57.349948 27501 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:17:57.355228 27713 tablet_service.cc:1467] Processing CreateTablet for tablet 01062855d3bc43cd930d3b1685751d03 (DEFAULT_TABLE table=test-table [id=6ae4caa73a75465eb8ceada4839bc20b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:57.355259 27922 tablet_service.cc:1467] Processing CreateTablet for tablet 01062855d3bc43cd930d3b1685751d03 (DEFAULT_TABLE table=test-table [id=6ae4caa73a75465eb8ceada4839bc20b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:57.355299 28119 tablet_service.cc:1467] Processing CreateTablet for tablet 01062855d3bc43cd930d3b1685751d03 (DEFAULT_TABLE table=test-table [id=6ae4caa73a75465eb8ceada4839bc20b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:57.355598 27922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01062855d3bc43cd930d3b1685751d03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:57.355599 27713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01062855d3bc43cd930d3b1685751d03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:57.355598 28119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01062855d3bc43cd930d3b1685751d03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:57.357877 28384 tablet_bootstrap.cc:492] T 01062855d3bc43cd930d3b1685751d03 P 73a70177480547aebc6eddbbde64eefd: Bootstrap starting.
I20260808 02:17:57.358363 28385 tablet_bootstrap.cc:492] T 01062855d3bc43cd930d3b1685751d03 P 477d4996df90406180e96af5e56bdd6e: Bootstrap starting.
I20260808 02:17:57.358587 28386 tablet_bootstrap.cc:492] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85: Bootstrap starting.
I20260808 02:17:57.359176 28384 tablet_bootstrap.cc:654] T 01062855d3bc43cd930d3b1685751d03 P 73a70177480547aebc6eddbbde64eefd: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:57.359244 28385 tablet_bootstrap.cc:654] T 01062855d3bc43cd930d3b1685751d03 P 477d4996df90406180e96af5e56bdd6e: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:57.359745 28386 tablet_bootstrap.cc:654] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:57.360608 28384 tablet_bootstrap.cc:492] T 01062855d3bc43cd930d3b1685751d03 P 73a70177480547aebc6eddbbde64eefd: No bootstrap required, opened a new log
I20260808 02:17:57.360613 28385 tablet_bootstrap.cc:492] T 01062855d3bc43cd930d3b1685751d03 P 477d4996df90406180e96af5e56bdd6e: No bootstrap required, opened a new log
I20260808 02:17:57.360713 28385 ts_tablet_manager.cc:1397] T 01062855d3bc43cd930d3b1685751d03 P 477d4996df90406180e96af5e56bdd6e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:17:57.360713 28384 ts_tablet_manager.cc:1397] T 01062855d3bc43cd930d3b1685751d03 P 73a70177480547aebc6eddbbde64eefd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:17:57.361181 28385 raft_consensus.cc:348] T 01062855d3bc43cd930d3b1685751d03 P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:57.361181 28384 raft_consensus.cc:348] T 01062855d3bc43cd930d3b1685751d03 P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:57.361313 28384 raft_consensus.cc:374] T 01062855d3bc43cd930d3b1685751d03 P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:57.361313 28385 raft_consensus.cc:374] T 01062855d3bc43cd930d3b1685751d03 P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:57.361347 28385 raft_consensus.cc:729] T 01062855d3bc43cd930d3b1685751d03 P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 477d4996df90406180e96af5e56bdd6e, State: Initialized, Role: FOLLOWER
I20260808 02:17:57.361347 28384 raft_consensus.cc:729] T 01062855d3bc43cd930d3b1685751d03 P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73a70177480547aebc6eddbbde64eefd, State: Initialized, Role: FOLLOWER
I20260808 02:17:57.361517 28385 consensus_queue.cc:260] T 01062855d3bc43cd930d3b1685751d03 P 477d4996df90406180e96af5e56bdd6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:57.361517 28384 consensus_queue.cc:260] T 01062855d3bc43cd930d3b1685751d03 P 73a70177480547aebc6eddbbde64eefd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:57.361840 28385 ts_tablet_manager.cc:1428] T 01062855d3bc43cd930d3b1685751d03 P 477d4996df90406180e96af5e56bdd6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:17:57.362434 28384 ts_tablet_manager.cc:1428] T 01062855d3bc43cd930d3b1685751d03 P 73a70177480547aebc6eddbbde64eefd: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:17:57.363143 28386 tablet_bootstrap.cc:492] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85: No bootstrap required, opened a new log
I20260808 02:17:57.363273 28386 ts_tablet_manager.cc:1397] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 02:17:57.363787 28386 raft_consensus.cc:348] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:57.363922 28386 raft_consensus.cc:374] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:57.363963 28386 raft_consensus.cc:729] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aed7f68aa194ebba090205fee848e85, State: Initialized, Role: FOLLOWER
I20260808 02:17:57.364108 28386 consensus_queue.cc:260] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:57.364512 28386 ts_tablet_manager.cc:1428] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:17:57.445801 28394 raft_consensus.cc:482] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:17:57.445989 28394 raft_consensus.cc:504] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:57.447274 28394 leader_election.cc:288] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73a70177480547aebc6eddbbde64eefd (127.24.79.194:46423), 477d4996df90406180e96af5e56bdd6e (127.24.79.195:44031)
I20260808 02:17:57.452108 28396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:57.447671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:38649 (local address 127.24.79.195:44031)
0808 02:17:57.448060 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:57.448070 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:57.448266 (+ 196us) server_negotiation.cc:407] Connection header received
0808 02:17:57.448395 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:57.448399 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:57.448453 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:57.448534 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:57.448893 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:57.449713 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:57.451014 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:57.451445 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:57.451571 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:57.451634 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:57.451867 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 02:17:57.451961 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":128,"threads_started":1}
I20260808 02:17:57.452108 28397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:57.447676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:38237 (local address 127.24.79.194:46423)
0808 02:17:57.447998 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:57.448006 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:57.448386 (+ 380us) server_negotiation.cc:407] Connection header received
0808 02:17:57.448507 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:57.448510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:57.448598 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:57.448692 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:17:57.449100 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:57.449705 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:57.451013 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:57.451457 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:57.451572 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:57.451634 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:57.451864 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 02:17:57.451947 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":122,"threads_started":1}
I20260808 02:17:57.452167 28398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:57.447562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:44031 (local address 127.24.79.193:38649)
0808 02:17:57.448167 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:57.448179 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:57.448292 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:57.448559 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:57.448562 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:57.448579 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:57.448744 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:57.448752 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:57.449860 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:57.449864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:57.450856 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:57.450867 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:57.451571 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:57.451575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:57.451725 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:57.451758 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 02:17:57.451829 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 02:17:57.451999 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"mutex_wait_us":87,"thread_start_us":202,"threads_started":1}
I20260808 02:17:57.452178 28395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:57.447564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:46423 (local address 127.24.79.193:38237)
0808 02:17:57.448254 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:57.448269 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:57.448406 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:57.448741 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:57.448743 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:57.448762 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:17:57.448897 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:57.448903 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:57.449848 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:57.449852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:57.450846 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:57.450858 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:57.451577 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:57.451580 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:57.451728 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:57.451754 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 02:17:57.451824 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 02:17:57.452011 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"mutex_wait_us":455,"thread_start_us":104,"threads_started":1}
I20260808 02:17:57.452702 27952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01062855d3bc43cd930d3b1685751d03" candidate_uuid: "8aed7f68aa194ebba090205fee848e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73a70177480547aebc6eddbbde64eefd" is_pre_election: true
I20260808 02:17:57.452886 27952 raft_consensus.cc:2454] T 01062855d3bc43cd930d3b1685751d03 P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8aed7f68aa194ebba090205fee848e85 in term 0.
I20260808 02:17:57.453022 28155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01062855d3bc43cd930d3b1685751d03" candidate_uuid: "8aed7f68aa194ebba090205fee848e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477d4996df90406180e96af5e56bdd6e" is_pre_election: true
I20260808 02:17:57.453174 28155 raft_consensus.cc:2454] T 01062855d3bc43cd930d3b1685751d03 P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8aed7f68aa194ebba090205fee848e85 in term 0.
I20260808 02:17:57.453348 27645 leader_election.cc:302] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73a70177480547aebc6eddbbde64eefd, 8aed7f68aa194ebba090205fee848e85; no voters:
I20260808 02:17:57.453641 28394 raft_consensus.cc:2781] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:17:57.453696 28394 raft_consensus.cc:482] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:17:57.453720 28394 raft_consensus.cc:3037] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:57.454818 28394 raft_consensus.cc:504] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:57.455251 28394 leader_election.cc:288] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85 [CANDIDATE]: Term 1 election: Requested vote from peers 73a70177480547aebc6eddbbde64eefd (127.24.79.194:46423), 477d4996df90406180e96af5e56bdd6e (127.24.79.195:44031)
I20260808 02:17:57.455751 27952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01062855d3bc43cd930d3b1685751d03" candidate_uuid: "8aed7f68aa194ebba090205fee848e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73a70177480547aebc6eddbbde64eefd"
I20260808 02:17:57.455751 28155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01062855d3bc43cd930d3b1685751d03" candidate_uuid: "8aed7f68aa194ebba090205fee848e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477d4996df90406180e96af5e56bdd6e"
I20260808 02:17:57.455904 28155 raft_consensus.cc:3037] T 01062855d3bc43cd930d3b1685751d03 P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:57.455904 27952 raft_consensus.cc:3037] T 01062855d3bc43cd930d3b1685751d03 P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:57.457295 28155 raft_consensus.cc:2454] T 01062855d3bc43cd930d3b1685751d03 P 477d4996df90406180e96af5e56bdd6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8aed7f68aa194ebba090205fee848e85 in term 1.
I20260808 02:17:57.457295 27952 raft_consensus.cc:2454] T 01062855d3bc43cd930d3b1685751d03 P 73a70177480547aebc6eddbbde64eefd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8aed7f68aa194ebba090205fee848e85 in term 1.
I20260808 02:17:57.457698 27645 leader_election.cc:302] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 477d4996df90406180e96af5e56bdd6e, 73a70177480547aebc6eddbbde64eefd, 8aed7f68aa194ebba090205fee848e85; no voters:
I20260808 02:17:57.457901 28394 raft_consensus.cc:2781] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:57.458156 28394 raft_consensus.cc:686] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85 [term 1 LEADER]: Becoming Leader. State: Replica: 8aed7f68aa194ebba090205fee848e85, State: Running, Role: LEADER
I20260808 02:17:57.458439 28394 consensus_queue.cc:237] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:57.461326 27501 catalog_manager.cc:5495] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8aed7f68aa194ebba090205fee848e85 (127.24.79.193). New cstate: current_term: 1 leader_uuid: "8aed7f68aa194ebba090205fee848e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } health_report { overall_health: UNKNOWN } } }
I20260808 02:17:57.502990 28402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:57.498545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54850 (local address 127.24.79.193:45371)
0808 02:17:57.498842 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:57.498850 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:57.498863 (+ 13us) server_negotiation.cc:407] Connection header received
0808 02:17:57.498992 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:57.498994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:57.499056 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:57.499169 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:57.499442 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:57.500042 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:57.500993 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:57.501295 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:57.501402 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:57.501793 (+ 391us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:57.502335 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:57.502449 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:57.502658 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 02:17:57.502745 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":117,"threads_started":1}
I20260808 02:17:57.515352 28397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:57.511826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:40345 (local address 127.24.79.194:46423)
0808 02:17:57.511988 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:57.511992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:57.512087 (+ 95us) server_negotiation.cc:407] Connection header received
0808 02:17:57.512278 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:57.512281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:57.512336 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:57.512415 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:57.512660 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:57.513186 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:57.514122 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:57.514353 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:57.514441 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:57.514873 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:57.515001 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:57.515077 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:57.515162 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 02:17:57.515224 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 02:17:57.515403 28405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:57.511568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:46423 (local address 127.24.79.193:40345)
0808 02:17:57.512007 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:57.512020 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:57.512127 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:57.512431 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:57.512433 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:57.512439 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:17:57.512564 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:57.512568 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:57.513332 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:57.513335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:57.513992 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:57.514003 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:57.514515 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:57.514519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:57.514612 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:57.514674 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:17:57.515084 (+ 410us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:17:57.515090 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 02:17:57.515141 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 02:17:57.515259 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":172,"threads_started":1}
I20260808 02:17:57.522275 27952 raft_consensus.cc:1261] T 01062855d3bc43cd930d3b1685751d03 P 73a70177480547aebc6eddbbde64eefd [term 1 FOLLOWER]: Refusing update from remote peer 8aed7f68aa194ebba090205fee848e85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:57.522285 28155 raft_consensus.cc:1261] T 01062855d3bc43cd930d3b1685751d03 P 477d4996df90406180e96af5e56bdd6e [term 1 FOLLOWER]: Refusing update from remote peer 8aed7f68aa194ebba090205fee848e85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:57.522789 28394 consensus_queue.cc:1035] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:57.523077 28399 consensus_queue.cc:1035] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:57.542238 28402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:57.538679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:35909 (local address 127.24.79.193:45371)
0808 02:17:57.538825 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:57.538830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:57.539087 (+ 257us) server_negotiation.cc:407] Connection header received
0808 02:17:57.539203 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:57.539206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:57.539259 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:57.539321 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:57.539560 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:57.540098 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:57.541015 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:57.541337 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:57.541449 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:57.541749 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:57.541831 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:57.541888 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:57.542025 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 02:17:57.542090 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:17:57.542263 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:57.538627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:45371 (local address 127.24.79.194:35909)
0808 02:17:57.538988 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:57.539003 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:57.539116 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:57.539338 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:57.539340 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:57.539349 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:17:57.539459 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:57.539465 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:57.540214 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:57.540217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:57.540871 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:57.540882 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:57.541468 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:57.541472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:57.541574 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:57.541608 (+ 34us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:17:57.541918 (+ 310us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:17:57.541926 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 02:17:57.541996 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 02:17:57.542098 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":124,"threads_started":1}
I20260808 02:17:57.571316 27499 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:17:57.571521 27499 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:17:57.573961 27499 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1: Sending DeleteTablet for 3 replicas of tablet 01062855d3bc43cd930d3b1685751d03
I20260808 02:17:57.574632 27713 tablet_service.cc:1514] Processing DeleteTablet for tablet 01062855d3bc43cd930d3b1685751d03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:57 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:54834
I20260808 02:17:57.574792 28119 tablet_service.cc:1514] Processing DeleteTablet for tablet 01062855d3bc43cd930d3b1685751d03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:57 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:45840
I20260808 02:17:57.574891 28372 tablet_replica.cc:330] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85: stopping tablet replica
I20260808 02:17:57.575044 28372 raft_consensus.cc:2229] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:17:57.575085 28374 tablet_replica.cc:330] T 01062855d3bc43cd930d3b1685751d03 P 477d4996df90406180e96af5e56bdd6e: stopping tablet replica
I20260808 02:17:57.575209 28374 raft_consensus.cc:2229] T 01062855d3bc43cd930d3b1685751d03 P 477d4996df90406180e96af5e56bdd6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:17:57.575351 28374 raft_consensus.cc:2258] T 01062855d3bc43cd930d3b1685751d03 P 477d4996df90406180e96af5e56bdd6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:57.575404 28372 raft_consensus.cc:2258] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:57.575698 27922 tablet_service.cc:1514] Processing DeleteTablet for tablet 01062855d3bc43cd930d3b1685751d03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:57 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:59862
I20260808 02:17:57.576239 28372 ts_tablet_manager.cc:1905] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:17:57.576265 28374 ts_tablet_manager.cc:1905] T 01062855d3bc43cd930d3b1685751d03 P 477d4996df90406180e96af5e56bdd6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:17:57.578439 28374 ts_tablet_manager.cc:1918] T 01062855d3bc43cd930d3b1685751d03 P 477d4996df90406180e96af5e56bdd6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:17:57.578512 28374 log.cc:1196] T 01062855d3bc43cd930d3b1685751d03 P 477d4996df90406180e96af5e56bdd6e: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/01062855d3bc43cd930d3b1685751d03
I20260808 02:17:57.578729 28374 ts_tablet_manager.cc:1939] T 01062855d3bc43cd930d3b1685751d03 P 477d4996df90406180e96af5e56bdd6e: Deleting consensus metadata
I20260808 02:17:57.578969 28372 ts_tablet_manager.cc:1918] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:17:57.579051 28372 log.cc:1196] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-0/wal/wals/01062855d3bc43cd930d3b1685751d03
I20260808 02:17:57.579308 28372 ts_tablet_manager.cc:1939] T 01062855d3bc43cd930d3b1685751d03 P 8aed7f68aa194ebba090205fee848e85: Deleting consensus metadata
I20260808 02:17:57.579520 27478 catalog_manager.cc:4841] TS 477d4996df90406180e96af5e56bdd6e (127.24.79.195:44031): tablet 01062855d3bc43cd930d3b1685751d03 (table test-table [id=6ae4caa73a75465eb8ceada4839bc20b]) successfully deleted
I20260808 02:17:57.580016 28373 tablet_replica.cc:330] T 01062855d3bc43cd930d3b1685751d03 P 73a70177480547aebc6eddbbde64eefd: stopping tablet replica
I20260808 02:17:57.580066 27484 catalog_manager.cc:4841] TS 8aed7f68aa194ebba090205fee848e85 (127.24.79.193:45371): tablet 01062855d3bc43cd930d3b1685751d03 (table test-table [id=6ae4caa73a75465eb8ceada4839bc20b]) successfully deleted
I20260808 02:17:57.580170 28373 raft_consensus.cc:2229] T 01062855d3bc43cd930d3b1685751d03 P 73a70177480547aebc6eddbbde64eefd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:17:57.580299 28373 raft_consensus.cc:2258] T 01062855d3bc43cd930d3b1685751d03 P 73a70177480547aebc6eddbbde64eefd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:57.580493 27499 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:17:57.580947 27499 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:17:57.581197 28373 ts_tablet_manager.cc:1905] T 01062855d3bc43cd930d3b1685751d03 P 73a70177480547aebc6eddbbde64eefd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:17:57.583915 28373 ts_tablet_manager.cc:1918] T 01062855d3bc43cd930d3b1685751d03 P 73a70177480547aebc6eddbbde64eefd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:17:57.583997 28373 log.cc:1196] T 01062855d3bc43cd930d3b1685751d03 P 73a70177480547aebc6eddbbde64eefd: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/01062855d3bc43cd930d3b1685751d03
I20260808 02:17:57.584233 28373 ts_tablet_manager.cc:1939] T 01062855d3bc43cd930d3b1685751d03 P 73a70177480547aebc6eddbbde64eefd: Deleting consensus metadata
I20260808 02:17:57.584967 27482 catalog_manager.cc:4841] TS 73a70177480547aebc6eddbbde64eefd (127.24.79.194:46423): tablet 01062855d3bc43cd930d3b1685751d03 (table test-table [id=6ae4caa73a75465eb8ceada4839bc20b]) successfully deleted
I20260808 02:17:57.585763 27713 tablet_service.cc:1467] Processing CreateTablet for tablet 41df4e9ceb114761a5dfe22498b25206 (DEFAULT_TABLE table=test-table [id=0073aa2c4b0a4602aacd51cf1d48bf61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:57.585866 28119 tablet_service.cc:1467] Processing CreateTablet for tablet 41df4e9ceb114761a5dfe22498b25206 (DEFAULT_TABLE table=test-table [id=0073aa2c4b0a4602aacd51cf1d48bf61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:57.585866 27922 tablet_service.cc:1467] Processing CreateTablet for tablet 41df4e9ceb114761a5dfe22498b25206 (DEFAULT_TABLE table=test-table [id=0073aa2c4b0a4602aacd51cf1d48bf61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:17:57.586058 27713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41df4e9ceb114761a5dfe22498b25206. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:57.586122 27922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41df4e9ceb114761a5dfe22498b25206. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:57.586122 28119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41df4e9ceb114761a5dfe22498b25206. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:57.588428 28384 tablet_bootstrap.cc:492] T 41df4e9ceb114761a5dfe22498b25206 P 73a70177480547aebc6eddbbde64eefd: Bootstrap starting.
I20260808 02:17:57.588446 28385 tablet_bootstrap.cc:492] T 41df4e9ceb114761a5dfe22498b25206 P 477d4996df90406180e96af5e56bdd6e: Bootstrap starting.
I20260808 02:17:57.589319 28384 tablet_bootstrap.cc:654] T 41df4e9ceb114761a5dfe22498b25206 P 73a70177480547aebc6eddbbde64eefd: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:57.589334 28385 tablet_bootstrap.cc:654] T 41df4e9ceb114761a5dfe22498b25206 P 477d4996df90406180e96af5e56bdd6e: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:57.589931 28386 tablet_bootstrap.cc:492] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85: Bootstrap starting.
I20260808 02:17:57.590605 28384 tablet_bootstrap.cc:492] T 41df4e9ceb114761a5dfe22498b25206 P 73a70177480547aebc6eddbbde64eefd: No bootstrap required, opened a new log
I20260808 02:17:57.590690 28384 ts_tablet_manager.cc:1397] T 41df4e9ceb114761a5dfe22498b25206 P 73a70177480547aebc6eddbbde64eefd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:17:57.590889 28386 tablet_bootstrap.cc:654] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:57.591125 28384 raft_consensus.cc:348] T 41df4e9ceb114761a5dfe22498b25206 P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:57.591238 28384 raft_consensus.cc:374] T 41df4e9ceb114761a5dfe22498b25206 P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:57.591282 28384 raft_consensus.cc:729] T 41df4e9ceb114761a5dfe22498b25206 P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73a70177480547aebc6eddbbde64eefd, State: Initialized, Role: FOLLOWER
I20260808 02:17:57.591414 28384 consensus_queue.cc:260] T 41df4e9ceb114761a5dfe22498b25206 P 73a70177480547aebc6eddbbde64eefd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:57.591715 28384 ts_tablet_manager.cc:1428] T 41df4e9ceb114761a5dfe22498b25206 P 73a70177480547aebc6eddbbde64eefd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:17:57.592304 28386 tablet_bootstrap.cc:492] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85: No bootstrap required, opened a new log
I20260808 02:17:57.592399 28386 ts_tablet_manager.cc:1397] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 02:17:57.592489 28385 tablet_bootstrap.cc:492] T 41df4e9ceb114761a5dfe22498b25206 P 477d4996df90406180e96af5e56bdd6e: No bootstrap required, opened a new log
I20260808 02:17:57.592597 28385 ts_tablet_manager.cc:1397] T 41df4e9ceb114761a5dfe22498b25206 P 477d4996df90406180e96af5e56bdd6e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 02:17:57.592875 28386 raft_consensus.cc:348] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:57.593016 28386 raft_consensus.cc:374] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:57.593044 28386 raft_consensus.cc:729] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aed7f68aa194ebba090205fee848e85, State: Initialized, Role: FOLLOWER
I20260808 02:17:57.593021 28385 raft_consensus.cc:348] T 41df4e9ceb114761a5dfe22498b25206 P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:57.593132 28385 raft_consensus.cc:374] T 41df4e9ceb114761a5dfe22498b25206 P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:57.593163 28385 raft_consensus.cc:729] T 41df4e9ceb114761a5dfe22498b25206 P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 477d4996df90406180e96af5e56bdd6e, State: Initialized, Role: FOLLOWER
I20260808 02:17:57.593154 28386 consensus_queue.cc:260] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:57.593290 28385 consensus_queue.cc:260] T 41df4e9ceb114761a5dfe22498b25206 P 477d4996df90406180e96af5e56bdd6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:57.593393 28386 ts_tablet_manager.cc:1428] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:17:57.593546 28385 ts_tablet_manager.cc:1428] T 41df4e9ceb114761a5dfe22498b25206 P 477d4996df90406180e96af5e56bdd6e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 02:17:57.631546 28399 raft_consensus.cc:482] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:17:57.631712 28399 raft_consensus.cc:504] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:57.632200 28399 leader_election.cc:288] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73a70177480547aebc6eddbbde64eefd (127.24.79.194:46423), 477d4996df90406180e96af5e56bdd6e (127.24.79.195:44031)
I20260808 02:17:57.632670 27952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41df4e9ceb114761a5dfe22498b25206" candidate_uuid: "8aed7f68aa194ebba090205fee848e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73a70177480547aebc6eddbbde64eefd" is_pre_election: true
I20260808 02:17:57.632730 28155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41df4e9ceb114761a5dfe22498b25206" candidate_uuid: "8aed7f68aa194ebba090205fee848e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477d4996df90406180e96af5e56bdd6e" is_pre_election: true
I20260808 02:17:57.632867 28155 raft_consensus.cc:2454] T 41df4e9ceb114761a5dfe22498b25206 P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8aed7f68aa194ebba090205fee848e85 in term 0.
I20260808 02:17:57.632869 27952 raft_consensus.cc:2454] T 41df4e9ceb114761a5dfe22498b25206 P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8aed7f68aa194ebba090205fee848e85 in term 0.
I20260808 02:17:57.633198 27645 leader_election.cc:302] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73a70177480547aebc6eddbbde64eefd, 8aed7f68aa194ebba090205fee848e85; no voters:
I20260808 02:17:57.633451 28399 raft_consensus.cc:2781] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:17:57.633518 28399 raft_consensus.cc:482] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:17:57.633566 28399 raft_consensus.cc:3037] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:57.634766 28399 raft_consensus.cc:504] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:57.635170 28399 leader_election.cc:288] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85 [CANDIDATE]: Term 1 election: Requested vote from peers 73a70177480547aebc6eddbbde64eefd (127.24.79.194:46423), 477d4996df90406180e96af5e56bdd6e (127.24.79.195:44031)
I20260808 02:17:57.635536 28155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41df4e9ceb114761a5dfe22498b25206" candidate_uuid: "8aed7f68aa194ebba090205fee848e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477d4996df90406180e96af5e56bdd6e"
I20260808 02:17:57.635663 28155 raft_consensus.cc:3037] T 41df4e9ceb114761a5dfe22498b25206 P 477d4996df90406180e96af5e56bdd6e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:57.635603 27952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41df4e9ceb114761a5dfe22498b25206" candidate_uuid: "8aed7f68aa194ebba090205fee848e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73a70177480547aebc6eddbbde64eefd"
I20260808 02:17:57.635766 27952 raft_consensus.cc:3037] T 41df4e9ceb114761a5dfe22498b25206 P 73a70177480547aebc6eddbbde64eefd [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:57.636631 28155 raft_consensus.cc:2454] T 41df4e9ceb114761a5dfe22498b25206 P 477d4996df90406180e96af5e56bdd6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8aed7f68aa194ebba090205fee848e85 in term 1.
I20260808 02:17:57.636941 27642 leader_election.cc:302] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 477d4996df90406180e96af5e56bdd6e, 8aed7f68aa194ebba090205fee848e85; no voters:
I20260808 02:17:57.636989 27952 raft_consensus.cc:2454] T 41df4e9ceb114761a5dfe22498b25206 P 73a70177480547aebc6eddbbde64eefd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8aed7f68aa194ebba090205fee848e85 in term 1.
I20260808 02:17:57.637136 28399 raft_consensus.cc:2781] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:57.637245 28399 raft_consensus.cc:686] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85 [term 1 LEADER]: Becoming Leader. State: Replica: 8aed7f68aa194ebba090205fee848e85, State: Running, Role: LEADER
I20260808 02:17:57.637411 28399 consensus_queue.cc:237] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } }
I20260808 02:17:57.639616 27500 catalog_manager.cc:5495] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8aed7f68aa194ebba090205fee848e85 (127.24.79.193). New cstate: current_term: 1 leader_uuid: "8aed7f68aa194ebba090205fee848e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aed7f68aa194ebba090205fee848e85" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 } health_report { overall_health: UNKNOWN } } }
I20260808 02:17:57.671768 28156 raft_consensus.cc:1261] T 41df4e9ceb114761a5dfe22498b25206 P 477d4996df90406180e96af5e56bdd6e [term 1 FOLLOWER]: Refusing update from remote peer 8aed7f68aa194ebba090205fee848e85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:57.672293 28399 consensus_queue.cc:1035] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "477d4996df90406180e96af5e56bdd6e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:57.672701 27952 raft_consensus.cc:1261] T 41df4e9ceb114761a5dfe22498b25206 P 73a70177480547aebc6eddbbde64eefd [term 1 FOLLOWER]: Refusing update from remote peer 8aed7f68aa194ebba090205fee848e85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:57.673174 28399 consensus_queue.cc:1035] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73a70177480547aebc6eddbbde64eefd" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 02:17:57.689810 28341 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 02:17:59.251820 27501 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:17:59.252005 27501 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:17:59.254896 27501 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f7632fff81f5499a898c713ff58442d1: Sending DeleteTablet for 3 replicas of tablet 41df4e9ceb114761a5dfe22498b25206
I20260808 02:17:59.255836 27713 tablet_service.cc:1514] Processing DeleteTablet for tablet 41df4e9ceb114761a5dfe22498b25206 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:59 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:54834
I20260808 02:17:59.255889 27922 tablet_service.cc:1514] Processing DeleteTablet for tablet 41df4e9ceb114761a5dfe22498b25206 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:59 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:59862
I20260808 02:17:59.255877 28119 tablet_service.cc:1514] Processing DeleteTablet for tablet 41df4e9ceb114761a5dfe22498b25206 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:17:59 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:45840
I20260808 02:17:59.256335 28456 tablet_replica.cc:330] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85: stopping tablet replica
I20260808 02:17:59.256475 28456 raft_consensus.cc:2229] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:17:59.256669 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 27615
I20260808 02:17:59.256703 28456 raft_consensus.cc:2258] T 41df4e9ceb114761a5dfe22498b25206 P 8aed7f68aa194ebba090205fee848e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:59.271348 28459 tablet_replica.cc:330] T 41df4e9ceb114761a5dfe22498b25206 P 477d4996df90406180e96af5e56bdd6e: stopping tablet replica
I20260808 02:17:59.271557 28459 raft_consensus.cc:2229] T 41df4e9ceb114761a5dfe22498b25206 P 477d4996df90406180e96af5e56bdd6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:17:59.271746 28457 tablet_replica.cc:330] T 41df4e9ceb114761a5dfe22498b25206 P 73a70177480547aebc6eddbbde64eefd: stopping tablet replica
I20260808 02:17:59.271783 28459 raft_consensus.cc:2258] T 41df4e9ceb114761a5dfe22498b25206 P 477d4996df90406180e96af5e56bdd6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:59.271896 28457 raft_consensus.cc:2229] T 41df4e9ceb114761a5dfe22498b25206 P 73a70177480547aebc6eddbbde64eefd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:17:59.272042 28457 raft_consensus.cc:2258] T 41df4e9ceb114761a5dfe22498b25206 P 73a70177480547aebc6eddbbde64eefd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:17:59.272717 28459 ts_tablet_manager.cc:1905] T 41df4e9ceb114761a5dfe22498b25206 P 477d4996df90406180e96af5e56bdd6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:17:59.272763 28457 ts_tablet_manager.cc:1905] T 41df4e9ceb114761a5dfe22498b25206 P 73a70177480547aebc6eddbbde64eefd: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 02:17:59.275933 27484 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.79.193:45371 (error 108)
I20260808 02:17:59.276059 28459 ts_tablet_manager.cc:1918] T 41df4e9ceb114761a5dfe22498b25206 P 477d4996df90406180e96af5e56bdd6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:17:59.276059 28457 ts_tablet_manager.cc:1918] T 41df4e9ceb114761a5dfe22498b25206 P 73a70177480547aebc6eddbbde64eefd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:17:59.276162 28457 log.cc:1196] T 41df4e9ceb114761a5dfe22498b25206 P 73a70177480547aebc6eddbbde64eefd: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/41df4e9ceb114761a5dfe22498b25206
I20260808 02:17:59.276247 28459 log.cc:1196] T 41df4e9ceb114761a5dfe22498b25206 P 477d4996df90406180e96af5e56bdd6e: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/41df4e9ceb114761a5dfe22498b25206
I20260808 02:17:59.276527 28459 ts_tablet_manager.cc:1939] T 41df4e9ceb114761a5dfe22498b25206 P 477d4996df90406180e96af5e56bdd6e: Deleting consensus metadata
I20260808 02:17:59.276528 28457 ts_tablet_manager.cc:1939] T 41df4e9ceb114761a5dfe22498b25206 P 73a70177480547aebc6eddbbde64eefd: Deleting consensus metadata
I20260808 02:17:59.277369 27478 catalog_manager.cc:4841] TS 477d4996df90406180e96af5e56bdd6e (127.24.79.195:44031): tablet 41df4e9ceb114761a5dfe22498b25206 (table test-table [id=0073aa2c4b0a4602aacd51cf1d48bf61]) successfully deleted
I20260808 02:17:59.277364 27482 catalog_manager.cc:4841] TS 73a70177480547aebc6eddbbde64eefd (127.24.79.194:46423): tablet 41df4e9ceb114761a5dfe22498b25206 (table test-table [id=0073aa2c4b0a4602aacd51cf1d48bf61]) successfully deleted
I20260808 02:17:59.277557 28461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:59.276822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:45371 (local address 127.0.0.1:54856)
0808 02:17:59.277145 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:59.277386 (+ 241us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.193:45371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":84,"threads_started":1}
I20260808 02:17:59.277611 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 27822
W20260808 02:17:59.277685 27484 catalog_manager.cc:4584] TS 8aed7f68aa194ebba090205fee848e85 (127.24.79.193:45371): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 41df4e9ceb114761a5dfe22498b25206: Network error: Client connection negotiation failed: client connection to 127.24.79.193:45371: connect: Connection refused (error 111)
I20260808 02:17:59.293083 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 28029
I20260808 02:17:59.307981 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 27458
2026-08-08T02:17:59Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6555 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:59 dist-test-slave-1vmg krb5kdc[28465](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:59 dist-test-slave-1vmg krb5kdc[28465](info): set up 2 sockets
Aug 08 02:17:59 dist-test-slave-1vmg krb5kdc[28465](info): commencing operation
krb5kdc: starting...
W20260808 02:18:00.464620 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:18:00 dist-test-slave-1vmg krb5kdc[28465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:18:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:18:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:18:00.606554 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:41807
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:41807
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:00.731482 28484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:00.731825 28484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:00.731869 28484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:00.736209 28484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:18:00.736302 28484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:00.736322 28484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:00.736339 28484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:18:00.736353 28484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:18:00.742188 28484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:41807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:41807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:00.743532 28484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:00.744663 28484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:00.754200 28498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:00.754206 28493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:00.754200 28492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:00.754263 28484 server_base.cc:1034] running on GCE node
I20260808 02:18:00.755142 28484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:00.756440 28484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:00.757629 28484 hybrid_clock.cc:648] HybridClock initialized: now 1786155480757607 us; error 41 us; skew 500 ppm
Aug 08 02:18:00 dist-test-slave-1vmg krb5kdc[28465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155480, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:00.765867 28484 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:18:00.767098 28484 webserver.cc:460] Webserver started at http://127.24.79.254:40977/ using document root <none> and password file <none>
I20260808 02:18:00.767773 28484 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:00.767858 28484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:00.768065 28484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:00.770474 28484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "0c65572106e34386a9d547ec05b7a276"
format_stamp: "Formatted at 2026-08-08 02:18:00 on dist-test-slave-1vmg"
server_key: "88e8977f4ba01e8418bd9ef30fc3c1ca"
server_key_iv: "c6fd3468d943fdc54291f670e8385ab7"
server_key_version: "encryptionkey@0"
I20260808 02:18:00.771157 28484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "0c65572106e34386a9d547ec05b7a276"
format_stamp: "Formatted at 2026-08-08 02:18:00 on dist-test-slave-1vmg"
server_key: "88e8977f4ba01e8418bd9ef30fc3c1ca"
server_key_iv: "c6fd3468d943fdc54291f670e8385ab7"
server_key_version: "encryptionkey@0"
I20260808 02:18:00.775612 28484 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260808 02:18:00.778640 28510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:00.780243 28484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:00.780398 28484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "0c65572106e34386a9d547ec05b7a276"
format_stamp: "Formatted at 2026-08-08 02:18:00 on dist-test-slave-1vmg"
server_key: "88e8977f4ba01e8418bd9ef30fc3c1ca"
server_key_iv: "c6fd3468d943fdc54291f670e8385ab7"
server_key_version: "encryptionkey@0"
I20260808 02:18:00.780543 28484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:00.819314 28484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:00.823418 28484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:00.823652 28484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:00.832201 28484 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:41807
I20260808 02:18:00.832221 28583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:41807 every 8 connection(s)
I20260808 02:18:00.833578 28484 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:18:00.836023 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 28484
I20260808 02:18:00.836126 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:18:00.836467 24895 external_mini_cluster.cc:1428] Setting key a2c2bd55618a34ae3297b4d925e9ebe0
I20260808 02:18:00.837625 28584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:00.845643 28584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c65572106e34386a9d547ec05b7a276: Bootstrap starting.
Aug 08 02:18:00 dist-test-slave-1vmg krb5kdc[28465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:18:00.848578 28584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c65572106e34386a9d547ec05b7a276: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:00.849838 28584 log.cc:824] T 00000000000000000000000000000000 P 0c65572106e34386a9d547ec05b7a276: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:00.853259 28584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c65572106e34386a9d547ec05b7a276: No bootstrap required, opened a new log
I20260808 02:18:00.854598 28587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:00.838070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55712 (local address 127.24.79.254:41807)
0808 02:18:00.838579 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:00.838607 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:00.838656 (+ 49us) server_negotiation.cc:407] Connection header received
0808 02:18:00.839593 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:00.839621 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:00.840146 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:00.840575 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:00.841429 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:00.843029 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:00.844079 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:00.844871 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:00.845054 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:00.847869 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:00.847877 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:00.847899 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:00.847946 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:00.850348 (+ 2402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:00.851089 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:00.851096 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:00.851104 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:00.851280 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:00.851643 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:00.851646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:00.851647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:00.852333 (+ 686us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:00.852605 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:00.853028 (+ 423us) server_negotiation.cc:299] Negotiation successful
0808 02:18:00.853458 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":146,"threads_started":1}
I20260808 02:18:00.857916 28584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c65572106e34386a9d547ec05b7a276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c65572106e34386a9d547ec05b7a276" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41807 } }
I20260808 02:18:00.858203 28584 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c65572106e34386a9d547ec05b7a276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:00.858243 28584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c65572106e34386a9d547ec05b7a276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c65572106e34386a9d547ec05b7a276, State: Initialized, Role: FOLLOWER
I20260808 02:18:00.858644 28584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c65572106e34386a9d547ec05b7a276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c65572106e34386a9d547ec05b7a276" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41807 } }
I20260808 02:18:00.858779 28584 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c65572106e34386a9d547ec05b7a276 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:18:00.858829 28584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c65572106e34386a9d547ec05b7a276 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:18:00.858908 28584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c65572106e34386a9d547ec05b7a276 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:00.860019 28584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c65572106e34386a9d547ec05b7a276 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c65572106e34386a9d547ec05b7a276" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41807 } }
I20260808 02:18:00.860371 28584 leader_election.cc:302] T 00000000000000000000000000000000 P 0c65572106e34386a9d547ec05b7a276 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c65572106e34386a9d547ec05b7a276; no voters:
I20260808 02:18:00.860698 28584 leader_election.cc:288] T 00000000000000000000000000000000 P 0c65572106e34386a9d547ec05b7a276 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:18:00.860874 28590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c65572106e34386a9d547ec05b7a276 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:00.861107 28590 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c65572106e34386a9d547ec05b7a276 [term 1 LEADER]: Becoming Leader. State: Replica: 0c65572106e34386a9d547ec05b7a276, State: Running, Role: LEADER
I20260808 02:18:00.861564 28590 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c65572106e34386a9d547ec05b7a276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c65572106e34386a9d547ec05b7a276" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41807 } }
I20260808 02:18:00.862277 28584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c65572106e34386a9d547ec05b7a276 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:18:00.863440 28592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c65572106e34386a9d547ec05b7a276 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c65572106e34386a9d547ec05b7a276. Latest consensus state: current_term: 1 leader_uuid: "0c65572106e34386a9d547ec05b7a276" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c65572106e34386a9d547ec05b7a276" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41807 } } }
I20260808 02:18:00.863605 28592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c65572106e34386a9d547ec05b7a276 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:00.864012 28605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:18:00.863963 28591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c65572106e34386a9d547ec05b7a276 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c65572106e34386a9d547ec05b7a276" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c65572106e34386a9d547ec05b7a276" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41807 } } }
I20260808 02:18:00.864121 28591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c65572106e34386a9d547ec05b7a276 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:00.867662 28605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:18:00.874330 28605 catalog_manager.cc:1347] Generated new cluster ID: 1da6b18487fa4e4bb23821f6870d43e0
I20260808 02:18:00.874426 28605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:18:00.885392 28605 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:18:00.886587 28605 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:18:00.892983 28605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c65572106e34386a9d547ec05b7a276: Generated new TSK 0
I20260808 02:18:00.894073 28605 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.79.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
I20260808 02:18:00.935253 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:41807
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:01.065092 28626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:01.065409 28626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:01.065476 28626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:01.069805 28626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:01.069911 28626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:01.070000 28626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:18:01.075769 28626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:41807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:01.077069 28626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:01.078295 28626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:01.089074 28637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:01.089318 28633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:01.089347 28632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:01.089428 28626 server_base.cc:1034] running on GCE node
I20260808 02:18:01.090027 28626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:01.090817 28626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:01.092044 28626 hybrid_clock.cc:648] HybridClock initialized: now 1786155481092018 us; error 37 us; skew 500 ppm
Aug 08 02:18:01 dist-test-slave-1vmg krb5kdc[28465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155481, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:01.095391 28626 init.cc:377] Logged in from keytab as kudu/127.24.79.193@KRBTEST.COM (short username kudu)
I20260808 02:18:01.096513 28626 webserver.cc:460] Webserver started at http://127.24.79.193:34865/ using document root <none> and password file <none>
I20260808 02:18:01.097122 28626 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:01.097188 28626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:01.097378 28626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:01.099813 28626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "a694ac9ced5c4fefac910d0f6eaa89e3"
format_stamp: "Formatted at 2026-08-08 02:18:01 on dist-test-slave-1vmg"
server_key: "0b12a01f4631440bb347aaa60387e44d"
server_key_iv: "ca964ab5701f20d64a74a61d2927cd15"
server_key_version: "encryptionkey@0"
I20260808 02:18:01.100450 28626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "a694ac9ced5c4fefac910d0f6eaa89e3"
format_stamp: "Formatted at 2026-08-08 02:18:01 on dist-test-slave-1vmg"
server_key: "0b12a01f4631440bb347aaa60387e44d"
server_key_iv: "ca964ab5701f20d64a74a61d2927cd15"
server_key_version: "encryptionkey@0"
I20260808 02:18:01.104970 28626 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:18:01.107960 28645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:01.109390 28626 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:01.109531 28626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "a694ac9ced5c4fefac910d0f6eaa89e3"
format_stamp: "Formatted at 2026-08-08 02:18:01 on dist-test-slave-1vmg"
server_key: "0b12a01f4631440bb347aaa60387e44d"
server_key_iv: "ca964ab5701f20d64a74a61d2927cd15"
server_key_version: "encryptionkey@0"
I20260808 02:18:01.109687 28626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:01.133158 28626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:01.137221 28626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:01.137437 28626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:01.138151 28626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:01.139178 28626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:01.139235 28626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:01.139279 28626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:01.139294 28626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:01.148713 28626 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:40265
I20260808 02:18:01.148811 28802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:40265 every 8 connection(s)
I20260808 02:18:01.149763 28626 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:18:01.155680 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 28626
I20260808 02:18:01.155808 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:18:01.156083 24895 external_mini_cluster.cc:1428] Setting key 21388a356c1b6e21996d808c29adce67
Aug 08 02:18:01 dist-test-slave-1vmg krb5kdc[28465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155481, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:18:01.166860 28587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:01.152026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:37513 (local address 127.24.79.254:41807)
0808 02:18:01.152243 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:01.152247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:01.153168 (+ 921us) server_negotiation.cc:407] Connection header received
0808 02:18:01.154490 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:01.154494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:01.154547 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:01.154633 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:01.156117 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:01.156840 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:01.158434 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:01.158882 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:01.159022 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:01.162295 (+ 3273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:01.162302 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:01.162306 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:01.162344 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:01.164413 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:01.165061 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:01.165065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:01.165067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:01.165183 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:01.165518 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:01.165522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:01.165523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:01.165808 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:01.165931 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:01.166499 (+ 568us) server_negotiation.cc:299] Negotiation successful
0808 02:18:01.166672 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260808 02:18:01.167768 28808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:01.152402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:41807 (local address 127.24.79.193:37513)
0808 02:18:01.152954 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:01.153003 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:01.154248 (+ 1245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:01.154852 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:01.154860 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:01.155340 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:01.155935 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:01.155957 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:01.157001 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:01.157005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:01.158271 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:01.158283 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:01.159174 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:01.159178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:01.159364 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:01.160079 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:01.160105 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:01.161977 (+ 1872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:01.164570 (+ 2593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:01.164578 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:01.164596 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:01.164943 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:01.165275 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:01.165277 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:01.165278 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:01.165431 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:01.165913 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:01.165919 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:01.166223 (+ 304us) client_negotiation.cc:769] Sending connection context
0808 02:18:01.166414 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 02:18:01.166735 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":149,"threads_started":1}
WARNING: no policy specified for kudu/127.24.79.194@KRBTEST.COM; defaulting to no policy
I20260808 02:18:01.169238 28804 heartbeater.cc:347] Connected to a master server at 127.24.79.254:41807
I20260808 02:18:01.169540 28804 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:01.170163 28804 heartbeater.cc:511] Master 127.24.79.254:41807 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.194@KRBTEST.COM" created.
I20260808 02:18:01.172276 28534 ts_manager.cc:194] Registered new tserver with Master: a694ac9ced5c4fefac910d0f6eaa89e3 (127.24.79.193:40265)
I20260808 02:18:01.173836 28534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.193@KRBTEST.COM'} at 127.24.79.193:37513
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
I20260808 02:18:01.191360 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:41807
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:01.313038 28811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:01.313390 28811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:01.313444 28811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:01.317631 28811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:01.317710 28811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:01.317771 28811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:18:01.323479 28811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:41807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:01.324687 28811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:01.325973 28811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:01.338740 28819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:01.338775 28820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:01.339007 28822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:01.339685 28811 server_base.cc:1034] running on GCE node
I20260808 02:18:01.340266 28811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:01.341030 28811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:01.342271 28811 hybrid_clock.cc:648] HybridClock initialized: now 1786155481342248 us; error 38 us; skew 500 ppm
Aug 08 02:18:01 dist-test-slave-1vmg krb5kdc[28465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155481, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:01.345724 28811 init.cc:377] Logged in from keytab as kudu/127.24.79.194@KRBTEST.COM (short username kudu)
I20260808 02:18:01.346853 28811 webserver.cc:460] Webserver started at http://127.24.79.194:38995/ using document root <none> and password file <none>
I20260808 02:18:01.347548 28811 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:01.347623 28811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:01.347846 28811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:01.350239 28811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "bcdddf08f256400cb9918b8cf4b045f3"
format_stamp: "Formatted at 2026-08-08 02:18:01 on dist-test-slave-1vmg"
server_key: "f7a83243f16a63945978bd088dcbc2e5"
server_key_iv: "2e55f645a0ef8e40a760edbb2d49e27d"
server_key_version: "encryptionkey@0"
I20260808 02:18:01.351002 28811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "bcdddf08f256400cb9918b8cf4b045f3"
format_stamp: "Formatted at 2026-08-08 02:18:01 on dist-test-slave-1vmg"
server_key: "f7a83243f16a63945978bd088dcbc2e5"
server_key_iv: "2e55f645a0ef8e40a760edbb2d49e27d"
server_key_version: "encryptionkey@0"
I20260808 02:18:01.355505 28811 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:18:01.358539 28833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:01.360278 28811 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:01.360446 28811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "bcdddf08f256400cb9918b8cf4b045f3"
format_stamp: "Formatted at 2026-08-08 02:18:01 on dist-test-slave-1vmg"
server_key: "f7a83243f16a63945978bd088dcbc2e5"
server_key_iv: "2e55f645a0ef8e40a760edbb2d49e27d"
server_key_version: "encryptionkey@0"
I20260808 02:18:01.360595 28811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:01.380708 28811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:01.384359 28811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:01.384572 28811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:01.385252 28811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:01.386217 28811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:01.386283 28811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:01.386343 28811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:01.386371 28811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:01.396149 28811 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:38085
I20260808 02:18:01.396178 28990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:38085 every 8 connection(s)
I20260808 02:18:01.397137 28811 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:18:01.400300 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 28811
I20260808 02:18:01.400416 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:18:01.400698 24895 external_mini_cluster.cc:1428] Setting key dd821869db4049be73529722a7e1e8cf
Aug 08 02:18:01 dist-test-slave-1vmg krb5kdc[28465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155481, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.79.195@KRBTEST.COM; defaulting to no policy
I20260808 02:18:01.413720 28587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:01.399196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:51109 (local address 127.24.79.254:41807)
0808 02:18:01.399364 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:01.399369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:01.400410 (+ 1041us) server_negotiation.cc:407] Connection header received
0808 02:18:01.401841 (+ 1431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:01.401844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:01.401901 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:01.401975 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:01.403370 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:01.403905 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:01.405601 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:01.405938 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:01.406055 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:01.409213 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:01.409218 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:01.409221 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:01.409261 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:01.411075 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:01.411780 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:01.411785 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:01.411786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:01.411891 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:01.412344 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:01.412347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:01.412348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:01.412562 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:01.412648 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:01.413382 (+ 734us) server_negotiation.cc:299] Negotiation successful
0808 02:18:01.413554 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.195@KRBTEST.COM" created.
I20260808 02:18:01.414903 28994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:01.399626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:41807 (local address 127.24.79.194:51109)
0808 02:18:01.400221 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:01.400272 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:01.401601 (+ 1329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:01.402200 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:01.402208 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:01.402612 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:01.403211 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:01.403227 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:01.404046 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:01.404051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:01.405434 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:01.405447 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:01.406108 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:01.406111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:01.406276 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:01.407028 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:01.407048 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:01.408957 (+ 1909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:01.411260 (+ 2303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:01.411268 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:01.411285 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:01.411644 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:01.411995 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:01.411997 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:01.411999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:01.412231 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:01.412680 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:01.412688 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:01.413078 (+ 390us) client_negotiation.cc:769] Sending connection context
0808 02:18:01.413288 (+ 210us) client_negotiation.cc:240] Negotiation successful
0808 02:18:01.413709 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":141,"threads_started":1}
I20260808 02:18:01.416319 28992 heartbeater.cc:347] Connected to a master server at 127.24.79.254:41807
I20260808 02:18:01.416595 28992 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:01.417179 28992 heartbeater.cc:511] Master 127.24.79.254:41807 requested a full tablet report, sending...
I20260808 02:18:01.418411 28534 ts_manager.cc:194] Registered new tserver with Master: bcdddf08f256400cb9918b8cf4b045f3 (127.24.79.194:38085)
I20260808 02:18:01.419081 28534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.194@KRBTEST.COM'} at 127.24.79.194:51109
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
I20260808 02:18:01.435376 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:41807
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:01.560052 28997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:01.560401 28997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:01.560448 28997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:01.564937 28997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:01.565023 28997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:01.565105 28997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:18:01.571153 28997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:41807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:01.572364 28997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:01.573414 28997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:01.583813 29002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:01.583981 28997 server_base.cc:1034] running on GCE node
W20260808 02:18:01.583840 29003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:01.583820 29007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:01.584671 28997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:01.585436 28997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:01.586644 28997 hybrid_clock.cc:648] HybridClock initialized: now 1786155481586617 us; error 37 us; skew 500 ppm
Aug 08 02:18:01 dist-test-slave-1vmg krb5kdc[28465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155481, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:01.590060 28997 init.cc:377] Logged in from keytab as kudu/127.24.79.195@KRBTEST.COM (short username kudu)
I20260808 02:18:01.591184 28997 webserver.cc:460] Webserver started at http://127.24.79.195:36385/ using document root <none> and password file <none>
I20260808 02:18:01.591775 28997 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:01.591841 28997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:01.592041 28997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:01.594334 28997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "06007f9d57cb4a0d8db4d381b9bf5b34"
format_stamp: "Formatted at 2026-08-08 02:18:01 on dist-test-slave-1vmg"
server_key: "374be76daff6efb2bdf325c9c785341b"
server_key_iv: "66f4aa3e29e12c4aa39595d9ade8c51a"
server_key_version: "encryptionkey@0"
I20260808 02:18:01.595034 28997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "06007f9d57cb4a0d8db4d381b9bf5b34"
format_stamp: "Formatted at 2026-08-08 02:18:01 on dist-test-slave-1vmg"
server_key: "374be76daff6efb2bdf325c9c785341b"
server_key_iv: "66f4aa3e29e12c4aa39595d9ade8c51a"
server_key_version: "encryptionkey@0"
I20260808 02:18:01.599531 28997 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:18:01.602449 29016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:01.603942 28997 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:01.604077 28997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "06007f9d57cb4a0d8db4d381b9bf5b34"
format_stamp: "Formatted at 2026-08-08 02:18:01 on dist-test-slave-1vmg"
server_key: "374be76daff6efb2bdf325c9c785341b"
server_key_iv: "66f4aa3e29e12c4aa39595d9ade8c51a"
server_key_version: "encryptionkey@0"
I20260808 02:18:01.604192 28997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:01.627126 28997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:01.631108 28997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:01.631310 28997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:01.631981 28997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:01.632946 28997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:01.632999 28997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:01.633044 28997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:01.633059 28997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:01.642715 28997 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:41369
I20260808 02:18:01.642764 29168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:41369 every 8 connection(s)
I20260808 02:18:01.643784 28997 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:18:01.644356 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 28997
I20260808 02:18:01.644479 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:18:01.644822 24895 external_mini_cluster.cc:1428] Setting key 1d61cd4785dcc59897d90fe3edaf1e31
Aug 08 02:18:01 dist-test-slave-1vmg krb5kdc[28465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155481, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:18:01.659785 28587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:01.646069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:48631 (local address 127.24.79.254:41807)
0808 02:18:01.646202 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:01.646207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:01.647484 (+ 1277us) server_negotiation.cc:407] Connection header received
0808 02:18:01.648895 (+ 1411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:01.648899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:01.648961 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:01.649034 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:01.650377 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:01.650977 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:01.652344 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:01.652590 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:01.652686 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:01.655754 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:01.655758 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:01.655760 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:01.655790 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:01.657289 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:01.657915 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:01.657919 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:01.657921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:01.658018 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:01.658446 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:01.658449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:01.658450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:01.658720 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:01.658806 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:01.659456 (+ 650us) server_negotiation.cc:299] Negotiation successful
0808 02:18:01.659635 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 02:18:01.660784 29172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:01.646657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:41807 (local address 127.24.79.195:48631)
0808 02:18:01.647323 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:01.647390 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:01.648661 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:01.649241 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:01.649249 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:01.649660 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:01.650216 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:01.650233 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:01.651157 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:01.651160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:01.652190 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:01.652202 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:01.652793 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:01.652797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:01.652981 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:01.653765 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:01.653785 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:01.655531 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:01.657433 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:01.657440 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:01.657457 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:01.657784 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:01.658118 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:01.658121 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:01.658122 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:01.658338 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:01.658824 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:01.658832 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:01.659199 (+ 367us) client_negotiation.cc:769] Sending connection context
0808 02:18:01.659427 (+ 228us) client_negotiation.cc:240] Negotiation successful
0808 02:18:01.659858 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":200,"threads_started":1}
I20260808 02:18:01.662137 29169 heartbeater.cc:347] Connected to a master server at 127.24.79.254:41807
I20260808 02:18:01.662448 29169 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:01.663110 29169 heartbeater.cc:511] Master 127.24.79.254:41807 requested a full tablet report, sending...
I20260808 02:18:01.664487 28534 ts_manager.cc:194] Registered new tserver with Master: 06007f9d57cb4a0d8db4d381b9bf5b34 (127.24.79.195:41369)
I20260808 02:18:01.665146 28534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.195@KRBTEST.COM'} at 127.24.79.195:48631
I20260808 02:18:01.672338 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 02:18:01.689453 28587 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:01.687719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55718 (local address 127.24.79.254:41807)
0808 02:18:01.687894 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:01.687899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:01.688080 (+ 181us) server_negotiation.cc:407] Connection header received
0808 02:18:01.688236 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:01.688239 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:01.688289 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:01.688362 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:01.689290 (+ 928us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55718: BlockingRecv error: recv got EOF from 127.0.0.1:55718 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 02:18:01.690852 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 28626
I20260808 02:18:01.706856 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 28811
I20260808 02:18:01.735316 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 28997
I20260808 02:18:01.750591 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 28484
2026-08-08T02:18:01Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2454 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:18:01 dist-test-slave-1vmg krb5kdc[29190](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:18:01 dist-test-slave-1vmg krb5kdc[29190](info): set up 2 sockets
Aug 08 02:18:01 dist-test-slave-1vmg krb5kdc[29190](info): commencing operation
krb5kdc: starting...
W20260808 02:18:02.911598 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:18:03 dist-test-slave-1vmg krb5kdc[29190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155483, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:18:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:18:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:18:03.059382 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:41165
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:44563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:41165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:03.181870 29208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:03.182188 29208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:03.182231 29208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:03.186369 29208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:18:03.186458 29208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:03.186477 29208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:03.186493 29208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:18:03.186507 29208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:18:03.192189 29208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:44563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:41165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:41165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:03.193411 29208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:03.194537 29208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:03.203665 29220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:03.203665 29217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:03.203696 29214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:03.204169 29208 server_base.cc:1034] running on GCE node
I20260808 02:18:03.204821 29208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:03.206032 29208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:03.207280 29208 hybrid_clock.cc:648] HybridClock initialized: now 1786155483207258 us; error 36 us; skew 500 ppm
Aug 08 02:18:03 dist-test-slave-1vmg krb5kdc[29190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155483, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:03.210301 29208 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:18:03.211393 29208 webserver.cc:460] Webserver started at http://127.24.79.254:43457/ using document root <none> and password file <none>
I20260808 02:18:03.211967 29208 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:03.212047 29208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:03.212353 29208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:03.214550 29208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "11d0f5353f6e4a6fb7edada0da039f71"
format_stamp: "Formatted at 2026-08-08 02:18:03 on dist-test-slave-1vmg"
server_key: "f91c06fb310d86e88f8a55c1d933aae4"
server_key_iv: "a47f7fd159ec0b81ebd8ff0368be7f3c"
server_key_version: "encryptionkey@0"
I20260808 02:18:03.215204 29208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "11d0f5353f6e4a6fb7edada0da039f71"
format_stamp: "Formatted at 2026-08-08 02:18:03 on dist-test-slave-1vmg"
server_key: "f91c06fb310d86e88f8a55c1d933aae4"
server_key_iv: "a47f7fd159ec0b81ebd8ff0368be7f3c"
server_key_version: "encryptionkey@0"
I20260808 02:18:03.219489 29208 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:18:03.222292 29232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:03.223783 29208 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:03.223933 29208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "11d0f5353f6e4a6fb7edada0da039f71"
format_stamp: "Formatted at 2026-08-08 02:18:03 on dist-test-slave-1vmg"
server_key: "f91c06fb310d86e88f8a55c1d933aae4"
server_key_iv: "a47f7fd159ec0b81ebd8ff0368be7f3c"
server_key_version: "encryptionkey@0"
I20260808 02:18:03.224051 29208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:03.238222 29208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:03.241689 29208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:03.241925 29208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:03.249372 29208 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:41165
I20260808 02:18:03.249760 29301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:41165 every 8 connection(s)
I20260808 02:18:03.250486 29208 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:18:03.253791 29302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:03.258051 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 29208
I20260808 02:18:03.258167 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:18:03.258446 24895 external_mini_cluster.cc:1428] Setting key d3362cd11b27acc2a5a07febf31980ce
I20260808 02:18:03.260622 29302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11d0f5353f6e4a6fb7edada0da039f71: Bootstrap starting.
I20260808 02:18:03.263489 29302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11d0f5353f6e4a6fb7edada0da039f71: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:03.264339 29302 log.cc:824] T 00000000000000000000000000000000 P 11d0f5353f6e4a6fb7edada0da039f71: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:03.266947 29302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11d0f5353f6e4a6fb7edada0da039f71: No bootstrap required, opened a new log
Aug 08 02:18:03 dist-test-slave-1vmg krb5kdc[29190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155483, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:18:03.271895 29302 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11d0f5353f6e4a6fb7edada0da039f71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11d0f5353f6e4a6fb7edada0da039f71" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41165 } }
I20260808 02:18:03.272267 29302 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11d0f5353f6e4a6fb7edada0da039f71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:03.272336 29302 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11d0f5353f6e4a6fb7edada0da039f71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11d0f5353f6e4a6fb7edada0da039f71, State: Initialized, Role: FOLLOWER
I20260808 02:18:03.272831 29302 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11d0f5353f6e4a6fb7edada0da039f71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11d0f5353f6e4a6fb7edada0da039f71" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41165 } }
I20260808 02:18:03.272976 29302 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11d0f5353f6e4a6fb7edada0da039f71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:18:03.273049 29302 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11d0f5353f6e4a6fb7edada0da039f71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:18:03.273162 29302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11d0f5353f6e4a6fb7edada0da039f71 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:03.274286 29302 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11d0f5353f6e4a6fb7edada0da039f71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11d0f5353f6e4a6fb7edada0da039f71" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41165 } }
I20260808 02:18:03.274657 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:03.260053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41082 (local address 127.24.79.254:41165)
0808 02:18:03.260529 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:03.260546 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:03.260585 (+ 39us) server_negotiation.cc:407] Connection header received
0808 02:18:03.261307 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:03.261329 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:03.261698 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:03.262016 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:03.263072 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:03.264188 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:03.265282 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:03.265798 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:03.265935 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:03.269081 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:03.269087 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:03.269110 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:03.269154 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:03.271044 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:03.271737 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:03.271744 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:03.271750 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:03.271896 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:03.272268 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:03.272272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:03.272274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:03.272869 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:03.273106 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:03.273420 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 02:18:03.273763 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":138,"threads_started":1}
I20260808 02:18:03.274657 29302 leader_election.cc:302] T 00000000000000000000000000000000 P 11d0f5353f6e4a6fb7edada0da039f71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11d0f5353f6e4a6fb7edada0da039f71; no voters:
I20260808 02:18:03.275166 29302 leader_election.cc:288] T 00000000000000000000000000000000 P 11d0f5353f6e4a6fb7edada0da039f71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:18:03.275274 29312 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11d0f5353f6e4a6fb7edada0da039f71 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:03.275452 29312 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11d0f5353f6e4a6fb7edada0da039f71 [term 1 LEADER]: Becoming Leader. State: Replica: 11d0f5353f6e4a6fb7edada0da039f71, State: Running, Role: LEADER
I20260808 02:18:03.275804 29312 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11d0f5353f6e4a6fb7edada0da039f71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11d0f5353f6e4a6fb7edada0da039f71" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41165 } }
I20260808 02:18:03.276654 29302 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11d0f5353f6e4a6fb7edada0da039f71 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:18:03.277626 29314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11d0f5353f6e4a6fb7edada0da039f71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11d0f5353f6e4a6fb7edada0da039f71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11d0f5353f6e4a6fb7edada0da039f71" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41165 } } }
I20260808 02:18:03.277774 29314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11d0f5353f6e4a6fb7edada0da039f71 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:03.278120 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:18:03.278437 29326 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:18:03.279256 29315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11d0f5353f6e4a6fb7edada0da039f71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11d0f5353f6e4a6fb7edada0da039f71. Latest consensus state: current_term: 1 leader_uuid: "11d0f5353f6e4a6fb7edada0da039f71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11d0f5353f6e4a6fb7edada0da039f71" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41165 } } }
I20260808 02:18:03.279507 29315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11d0f5353f6e4a6fb7edada0da039f71 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:03.282063 29326 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:18:03.288866 29326 catalog_manager.cc:1347] Generated new cluster ID: 923da7e50a1c43fbb2b979eb9d762c8a
I20260808 02:18:03.288961 29326 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:18:03.303794 29326 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:18:03.305011 29326 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:18:03.309734 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:03.303250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41092 (local address 127.24.79.254:41165)
0808 02:18:03.303368 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:03.303373 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:03.303659 (+ 286us) server_negotiation.cc:407] Connection header received
0808 02:18:03.303855 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:03.303859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:03.303916 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:03.304008 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:03.304703 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:03.305791 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:03.307062 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:03.307440 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:03.307546 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:03.308580 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:03.308592 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:03.308595 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:18:03.309195 (+ 600us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:03.309232 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:18:03.309254 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:03.309349 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:03.309482 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 02:18:03.309575 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 02:18:03.310295 29326 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11d0f5353f6e4a6fb7edada0da039f71: Generated new TSK 0
I20260808 02:18:03.311172 29326 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:18:03.319918 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 29208
2026-08-08T02:18:03Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1554 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:18:03 dist-test-slave-1vmg krb5kdc[29354](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:18:03 dist-test-slave-1vmg krb5kdc[29354](info): set up 2 sockets
Aug 08 02:18:03 dist-test-slave-1vmg krb5kdc[29354](info): commencing operation
krb5kdc: starting...
W20260808 02:18:04.460510 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:18:04 dist-test-slave-1vmg krb5kdc[29354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:18:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:18:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:18:04.603678 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:40249
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:33907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:40249
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:04.727806 29374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:04.728157 29374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:04.728224 29374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:04.732519 29374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:18:04.732627 29374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:04.732662 29374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:04.732690 29374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:18:04.732723 29374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:18:04.738592 29374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:33907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:40249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:40249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:04.739972 29374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:04.741308 29374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:04.750654 29382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:04.750649 29381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:04.750648 29385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:04.750952 29374 server_base.cc:1034] running on GCE node
I20260808 02:18:04.751678 29374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:04.752909 29374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:04.754089 29374 hybrid_clock.cc:648] HybridClock initialized: now 1786155484754075 us; error 35 us; skew 500 ppm
Aug 08 02:18:04 dist-test-slave-1vmg krb5kdc[29354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155484, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:04.757185 29374 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:18:04.758239 29374 webserver.cc:460] Webserver started at http://127.24.79.254:39289/ using document root <none> and password file <none>
I20260808 02:18:04.758827 29374 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:04.758898 29374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:04.759181 29374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:04.761440 29374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "0bd52d85f4b948deaf7ba5fd8345e82c"
format_stamp: "Formatted at 2026-08-08 02:18:04 on dist-test-slave-1vmg"
server_key: "137a55410a8a1e82ead8819e24ae7bdf"
server_key_iv: "35db36541fffa83dafa8645e5ec555b5"
server_key_version: "encryptionkey@0"
I20260808 02:18:04.762090 29374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "0bd52d85f4b948deaf7ba5fd8345e82c"
format_stamp: "Formatted at 2026-08-08 02:18:04 on dist-test-slave-1vmg"
server_key: "137a55410a8a1e82ead8819e24ae7bdf"
server_key_iv: "35db36541fffa83dafa8645e5ec555b5"
server_key_version: "encryptionkey@0"
I20260808 02:18:04.766604 29374 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:18:04.769694 29394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:04.771332 29374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:04.771484 29374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "0bd52d85f4b948deaf7ba5fd8345e82c"
format_stamp: "Formatted at 2026-08-08 02:18:04 on dist-test-slave-1vmg"
server_key: "137a55410a8a1e82ead8819e24ae7bdf"
server_key_iv: "35db36541fffa83dafa8645e5ec555b5"
server_key_version: "encryptionkey@0"
I20260808 02:18:04.771617 29374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:04.792743 29374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:04.796387 29374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:04.796608 29374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:04.804615 29374 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:40249
I20260808 02:18:04.804948 29467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:40249 every 8 connection(s)
I20260808 02:18:04.805776 29374 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:18:04.809108 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:04.812831 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 29374
I20260808 02:18:04.812991 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:18:04.813321 24895 external_mini_cluster.cc:1428] Setting key 39507f6b20a034a8c0f2abb40e8451f5
I20260808 02:18:04.816288 29468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bd52d85f4b948deaf7ba5fd8345e82c: Bootstrap starting.
I20260808 02:18:04.819173 29468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bd52d85f4b948deaf7ba5fd8345e82c: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:04.820278 29468 log.cc:824] T 00000000000000000000000000000000 P 0bd52d85f4b948deaf7ba5fd8345e82c: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:04.822906 29468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bd52d85f4b948deaf7ba5fd8345e82c: No bootstrap required, opened a new log
Aug 08 02:18:04 dist-test-slave-1vmg krb5kdc[29354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:18:04.827667 29468 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bd52d85f4b948deaf7ba5fd8345e82c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd52d85f4b948deaf7ba5fd8345e82c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 40249 } }
I20260808 02:18:04.827999 29468 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bd52d85f4b948deaf7ba5fd8345e82c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:04.828078 29468 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bd52d85f4b948deaf7ba5fd8345e82c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd52d85f4b948deaf7ba5fd8345e82c, State: Initialized, Role: FOLLOWER
I20260808 02:18:04.828639 29468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bd52d85f4b948deaf7ba5fd8345e82c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd52d85f4b948deaf7ba5fd8345e82c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 40249 } }
I20260808 02:18:04.828817 29468 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bd52d85f4b948deaf7ba5fd8345e82c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:18:04.828881 29468 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bd52d85f4b948deaf7ba5fd8345e82c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:18:04.828977 29468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bd52d85f4b948deaf7ba5fd8345e82c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:04.830114 29473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:04.815034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53406 (local address 127.24.79.254:40249)
0808 02:18:04.815497 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:04.815516 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:04.815554 (+ 38us) server_negotiation.cc:407] Connection header received
0808 02:18:04.816222 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:04.816244 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:04.816643 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:04.816959 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:04.817949 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:04.819062 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:04.820317 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:04.821092 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:04.821285 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:04.824068 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:04.824075 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:04.824095 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:04.824132 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:04.826224 (+ 2092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:04.826783 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:04.826791 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:04.826798 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:04.827016 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:04.827383 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:04.827386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:04.827388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:04.827989 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:04.828211 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:04.828609 (+ 398us) server_negotiation.cc:299] Negotiation successful
0808 02:18:04.828980 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":130,"threads_started":1}
I20260808 02:18:04.830325 29468 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bd52d85f4b948deaf7ba5fd8345e82c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd52d85f4b948deaf7ba5fd8345e82c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 40249 } }
I20260808 02:18:04.830737 29468 leader_election.cc:302] T 00000000000000000000000000000000 P 0bd52d85f4b948deaf7ba5fd8345e82c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bd52d85f4b948deaf7ba5fd8345e82c; no voters:
I20260808 02:18:04.831156 29468 leader_election.cc:288] T 00000000000000000000000000000000 P 0bd52d85f4b948deaf7ba5fd8345e82c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:18:04.831278 29476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bd52d85f4b948deaf7ba5fd8345e82c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:04.831563 29476 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bd52d85f4b948deaf7ba5fd8345e82c [term 1 LEADER]: Becoming Leader. State: Replica: 0bd52d85f4b948deaf7ba5fd8345e82c, State: Running, Role: LEADER
I20260808 02:18:04.831923 29476 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bd52d85f4b948deaf7ba5fd8345e82c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd52d85f4b948deaf7ba5fd8345e82c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 40249 } }
I20260808 02:18:04.832631 29468 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bd52d85f4b948deaf7ba5fd8345e82c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:18:04.833411 29479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd52d85f4b948deaf7ba5fd8345e82c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bd52d85f4b948deaf7ba5fd8345e82c. Latest consensus state: current_term: 1 leader_uuid: "0bd52d85f4b948deaf7ba5fd8345e82c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd52d85f4b948deaf7ba5fd8345e82c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 40249 } } }
I20260808 02:18:04.833563 29479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd52d85f4b948deaf7ba5fd8345e82c [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:04.833809 29481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd52d85f4b948deaf7ba5fd8345e82c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bd52d85f4b948deaf7ba5fd8345e82c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd52d85f4b948deaf7ba5fd8345e82c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 40249 } } }
I20260808 02:18:04.833927 29481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd52d85f4b948deaf7ba5fd8345e82c [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:04.834004 29490 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:18:04.837620 29490 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:18:04.843979 29490 catalog_manager.cc:1347] Generated new cluster ID: 448dbe72469a448e83e14188c7781803
I20260808 02:18:04.844084 29490 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:18:04.852866 29490 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:18:04.854287 29490 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:18:04.861268 29490 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bd52d85f4b948deaf7ba5fd8345e82c: Generated new TSK 0
I20260808 02:18:04.862303 29490 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.79.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
I20260808 02:18:04.901353 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:40249
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:33907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:05.025434 29508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:05.025754 29508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:05.025801 29508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:05.030033 29508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:05.030123 29508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:05.030189 29508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:18:05.036098 29508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:33907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:40249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:05.037396 29508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:05.038538 29508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:05.049578 29517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:05.049537 29515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:05.049948 29520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:05.049821 29508 server_base.cc:1034] running on GCE node
I20260808 02:18:05.050431 29508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:05.051205 29508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:05.052438 29508 hybrid_clock.cc:648] HybridClock initialized: now 1786155485052415 us; error 38 us; skew 500 ppm
Aug 08 02:18:05 dist-test-slave-1vmg krb5kdc[29354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155485, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:05.055799 29508 init.cc:377] Logged in from keytab as kudu/127.24.79.193@KRBTEST.COM (short username kudu)
I20260808 02:18:05.056911 29508 webserver.cc:460] Webserver started at http://127.24.79.193:39309/ using document root <none> and password file <none>
I20260808 02:18:05.057534 29508 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:05.057618 29508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:05.057849 29508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:05.060437 29508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "e76890bcc84d4a75a6306d373faa5542"
format_stamp: "Formatted at 2026-08-08 02:18:05 on dist-test-slave-1vmg"
server_key: "98837b7c922cb5a2ccce8d274280873d"
server_key_iv: "d473db73465d0f126c8fc0777d7fd1e5"
server_key_version: "encryptionkey@0"
I20260808 02:18:05.061103 29508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "e76890bcc84d4a75a6306d373faa5542"
format_stamp: "Formatted at 2026-08-08 02:18:05 on dist-test-slave-1vmg"
server_key: "98837b7c922cb5a2ccce8d274280873d"
server_key_iv: "d473db73465d0f126c8fc0777d7fd1e5"
server_key_version: "encryptionkey@0"
I20260808 02:18:05.065747 29508 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:18:05.068815 29529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:05.070354 29508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:18:05.070509 29508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "e76890bcc84d4a75a6306d373faa5542"
format_stamp: "Formatted at 2026-08-08 02:18:05 on dist-test-slave-1vmg"
server_key: "98837b7c922cb5a2ccce8d274280873d"
server_key_iv: "d473db73465d0f126c8fc0777d7fd1e5"
server_key_version: "encryptionkey@0"
I20260808 02:18:05.070628 29508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:05.086408 29508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:05.090281 29508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:05.090458 29508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:05.091122 29508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:05.092039 29508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:05.092094 29508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:05.092137 29508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:05.092157 29508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:05.101295 29508 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:39463
I20260808 02:18:05.101372 29697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:39463 every 8 connection(s)
I20260808 02:18:05.102306 29508 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:18:05.109764 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 29508
I20260808 02:18:05.109910 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:18:05.110251 24895 external_mini_cluster.cc:1428] Setting key b2a95156b8069f88e6e4a70d68aaad17
Aug 08 02:18:05 dist-test-slave-1vmg krb5kdc[29354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155485, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:18:05.119283 29473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:05.104520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:57037 (local address 127.24.79.254:40249)
0808 02:18:05.104654 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:05.104659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:05.105536 (+ 877us) server_negotiation.cc:407] Connection header received
0808 02:18:05.106908 (+ 1372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:05.106943 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:05.107009 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:05.107105 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:05.108404 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:05.108977 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:05.110651 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:05.111016 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:05.111115 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:05.114481 (+ 3366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:05.114486 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:05.114489 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:05.114529 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:05.116361 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:05.117056 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:05.117061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:05.117063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:05.117199 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:05.117649 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:05.117652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:05.117654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:05.117918 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:05.118028 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:05.118864 (+ 836us) server_negotiation.cc:299] Negotiation successful
0808 02:18:05.119075 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 02:18:05.120254 29700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:05.104852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:40249 (local address 127.24.79.193:57037)
0808 02:18:05.105358 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:05.105405 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:05.106653 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:05.107278 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:05.107286 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:05.107705 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:05.108233 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:05.108248 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:05.109112 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:05.109115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:05.110466 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:05.110481 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:05.111206 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:05.111211 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:05.111380 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:05.112075 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:05.112099 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:05.114196 (+ 2097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:05.116528 (+ 2332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:05.116535 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:05.116554 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:05.116916 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:05.117339 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:05.117342 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:05.117343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:05.117542 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:05.118080 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:05.118089 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:05.118498 (+ 409us) client_negotiation.cc:769] Sending connection context
0808 02:18:05.118763 (+ 265us) client_negotiation.cc:240] Negotiation successful
0808 02:18:05.119177 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":122,"threads_started":1}
I20260808 02:18:05.121788 29698 heartbeater.cc:347] Connected to a master server at 127.24.79.254:40249
I20260808 02:18:05.122081 29698 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.24.79.194@KRBTEST.COM; defaulting to no policy
I20260808 02:18:05.122714 29698 heartbeater.cc:511] Master 127.24.79.254:40249 requested a full tablet report, sending...
I20260808 02:18:05.124830 29413 ts_manager.cc:194] Registered new tserver with Master: e76890bcc84d4a75a6306d373faa5542 (127.24.79.193:39463)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.194@KRBTEST.COM" created.
I20260808 02:18:05.126245 29413 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.193@KRBTEST.COM'} at 127.24.79.193:57037
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
I20260808 02:18:05.146705 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:40249
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:33907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:05.270471 29703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:05.270803 29703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:05.270870 29703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:05.275190 29703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:05.275282 29703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:05.275372 29703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:18:05.281323 29703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:33907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:40249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:05.282594 29703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:05.283751 29703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:05.294325 29710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:05.294274 29708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:05.294353 29703 server_base.cc:1034] running on GCE node
W20260808 02:18:05.294272 29717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:05.295131 29703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:05.295887 29703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:05.297055 29703 hybrid_clock.cc:648] HybridClock initialized: now 1786155485297034 us; error 39 us; skew 500 ppm
Aug 08 02:18:05 dist-test-slave-1vmg krb5kdc[29354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155485, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:05.300186 29703 init.cc:377] Logged in from keytab as kudu/127.24.79.194@KRBTEST.COM (short username kudu)
I20260808 02:18:05.301227 29703 webserver.cc:460] Webserver started at http://127.24.79.194:39191/ using document root <none> and password file <none>
I20260808 02:18:05.301841 29703 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:05.301918 29703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:05.302150 29703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:05.304458 29703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "3c1dbce9cb1f465f9976e88b46b09f0f"
format_stamp: "Formatted at 2026-08-08 02:18:05 on dist-test-slave-1vmg"
server_key: "a1f95cfbd0be20137401463f1e9309a4"
server_key_iv: "b43031cd5719a63e6b67b23aa4d14d52"
server_key_version: "encryptionkey@0"
I20260808 02:18:05.305078 29703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "3c1dbce9cb1f465f9976e88b46b09f0f"
format_stamp: "Formatted at 2026-08-08 02:18:05 on dist-test-slave-1vmg"
server_key: "a1f95cfbd0be20137401463f1e9309a4"
server_key_iv: "b43031cd5719a63e6b67b23aa4d14d52"
server_key_version: "encryptionkey@0"
I20260808 02:18:05.309509 29703 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260808 02:18:05.312572 29726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:05.314066 29703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:18:05.314227 29703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "3c1dbce9cb1f465f9976e88b46b09f0f"
format_stamp: "Formatted at 2026-08-08 02:18:05 on dist-test-slave-1vmg"
server_key: "a1f95cfbd0be20137401463f1e9309a4"
server_key_iv: "b43031cd5719a63e6b67b23aa4d14d52"
server_key_version: "encryptionkey@0"
I20260808 02:18:05.314364 29703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:05.336259 29703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:05.340610 29703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:05.340850 29703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:05.341554 29703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:05.342512 29703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:05.342564 29703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:05.342620 29703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:05.342651 29703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:05.352291 29703 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:45049
I20260808 02:18:05.352324 29894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:45049 every 8 connection(s)
I20260808 02:18:05.353415 29703 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:18:05.355696 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 29703
I20260808 02:18:05.355830 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:18:05.356107 24895 external_mini_cluster.cc:1428] Setting key 8bd376d1fa940a395e2b6c1534b9238e
Aug 08 02:18:05 dist-test-slave-1vmg krb5kdc[29354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155485, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.79.195@KRBTEST.COM; defaulting to no policy
I20260808 02:18:05.369935 29473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:05.355545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:52465 (local address 127.24.79.254:40249)
0808 02:18:05.355716 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:05.355722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:05.356907 (+ 1185us) server_negotiation.cc:407] Connection header received
0808 02:18:05.358407 (+ 1500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:05.358411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:05.358470 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:05.358561 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:05.360149 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:05.360824 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:05.362146 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:05.362480 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:05.362622 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:05.365758 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:05.365764 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:05.365767 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:05.365803 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:05.367627 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:05.368289 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:05.368293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:05.368295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:05.368381 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:05.368718 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:05.368720 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:05.368721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:05.368916 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:05.368992 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:05.369665 (+ 673us) server_negotiation.cc:299] Negotiation successful
0808 02:18:05.369796 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.195@KRBTEST.COM" created.
I20260808 02:18:05.370905 29897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:05.356038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:40249 (local address 127.24.79.194:52465)
0808 02:18:05.356733 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:05.356795 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:05.358147 (+ 1352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:05.358767 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:05.358775 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:05.359249 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:05.359964 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:05.359986 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:05.361000 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:05.361004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:05.362000 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:05.362013 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:05.362687 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:05.362690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:05.362862 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:05.363586 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:05.363606 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:05.365510 (+ 1904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:05.367811 (+ 2301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:05.367819 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:05.367837 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:05.368150 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:05.368464 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:05.368467 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:05.368468 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:05.368645 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:05.368992 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:05.368999 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:05.369349 (+ 350us) client_negotiation.cc:769] Sending connection context
0808 02:18:05.369576 (+ 227us) client_negotiation.cc:240] Negotiation successful
0808 02:18:05.369939 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":143,"threads_started":1}
I20260808 02:18:05.372308 29895 heartbeater.cc:347] Connected to a master server at 127.24.79.254:40249
I20260808 02:18:05.372570 29895 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:05.373133 29895 heartbeater.cc:511] Master 127.24.79.254:40249 requested a full tablet report, sending...
I20260808 02:18:05.374351 29413 ts_manager.cc:194] Registered new tserver with Master: 3c1dbce9cb1f465f9976e88b46b09f0f (127.24.79.194:45049)
I20260808 02:18:05.375128 29413 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.194@KRBTEST.COM'} at 127.24.79.194:52465
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
I20260808 02:18:05.388000 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:40249
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:33907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:05.511720 29902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:05.512079 29902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:05.512147 29902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:05.516551 29902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:05.516655 29902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:05.516743 29902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:18:05.522666 29902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:33907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:40249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:05.523979 29902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:05.525120 29902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:05.535816 29912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:05.535816 29908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:05.535881 29909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:05.536098 29902 server_base.cc:1034] running on GCE node
I20260808 02:18:05.536718 29902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:05.537421 29902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:05.538661 29902 hybrid_clock.cc:648] HybridClock initialized: now 1786155485538622 us; error 50 us; skew 500 ppm
Aug 08 02:18:05 dist-test-slave-1vmg krb5kdc[29354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155485, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:05.542197 29902 init.cc:377] Logged in from keytab as kudu/127.24.79.195@KRBTEST.COM (short username kudu)
I20260808 02:18:05.543413 29902 webserver.cc:460] Webserver started at http://127.24.79.195:33193/ using document root <none> and password file <none>
I20260808 02:18:05.544088 29902 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:05.544179 29902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:05.544432 29902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:05.547050 29902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "0204a1f185854701b35b2c6ee21583ac"
format_stamp: "Formatted at 2026-08-08 02:18:05 on dist-test-slave-1vmg"
server_key: "340be0ff6190c2cbd0da4c7aa3960e7d"
server_key_iv: "ab0789ad27c251e5d2fbff60e9d8664e"
server_key_version: "encryptionkey@0"
I20260808 02:18:05.547914 29902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "0204a1f185854701b35b2c6ee21583ac"
format_stamp: "Formatted at 2026-08-08 02:18:05 on dist-test-slave-1vmg"
server_key: "340be0ff6190c2cbd0da4c7aa3960e7d"
server_key_iv: "ab0789ad27c251e5d2fbff60e9d8664e"
server_key_version: "encryptionkey@0"
I20260808 02:18:05.552556 29902 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:18:05.555824 29923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:05.557550 29902 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:18:05.557716 29902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "0204a1f185854701b35b2c6ee21583ac"
format_stamp: "Formatted at 2026-08-08 02:18:05 on dist-test-slave-1vmg"
server_key: "340be0ff6190c2cbd0da4c7aa3960e7d"
server_key_iv: "ab0789ad27c251e5d2fbff60e9d8664e"
server_key_version: "encryptionkey@0"
I20260808 02:18:05.557850 29902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:05.587483 29902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:05.591182 29902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:05.591398 29902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:05.592083 29902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:05.593022 29902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:05.593075 29902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:05.593140 29902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:05.593166 29902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:05.602403 29902 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:41799
I20260808 02:18:05.602550 30087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:41799 every 8 connection(s)
I20260808 02:18:05.603575 29902 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:18:05.606997 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 29902
I20260808 02:18:05.607136 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:18:05.607445 24895 external_mini_cluster.cc:1428] Setting key 1e21cad54bbae8e1faf0665089bc2457
Aug 08 02:18:05 dist-test-slave-1vmg krb5kdc[29354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155485, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:18:05.619099 29473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:05.605735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:54245 (local address 127.24.79.254:40249)
0808 02:18:05.605926 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:05.605931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:05.606952 (+ 1021us) server_negotiation.cc:407] Connection header received
0808 02:18:05.608623 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:05.608627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:05.608690 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:05.608771 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:05.610088 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:05.610642 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:05.611992 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:05.612233 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:05.612331 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:05.615138 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:05.615141 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:05.615150 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:05.615183 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:05.616663 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:05.617317 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:05.617321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:05.617322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:05.617414 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:05.617813 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:05.617816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:05.617817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:05.618031 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:05.618124 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:05.618738 (+ 614us) server_negotiation.cc:299] Negotiation successful
0808 02:18:05.618886 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 02:18:05.620044 30091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:05.606152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:40249 (local address 127.24.79.195:54245)
0808 02:18:05.606758 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:05.606807 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:05.608371 (+ 1564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:05.608969 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:05.608978 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:05.609380 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:05.609931 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:05.609947 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:05.610789 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:05.610792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:05.611838 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:05.611851 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:05.612403 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:05.612408 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:05.612596 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:05.613266 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:05.613284 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:05.614900 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:05.616829 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:05.616836 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:05.616853 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:05.617161 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:05.617512 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:05.617515 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:05.617518 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:05.617696 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:05.618143 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:05.618152 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:05.618469 (+ 317us) client_negotiation.cc:769] Sending connection context
0808 02:18:05.618655 (+ 186us) client_negotiation.cc:240] Negotiation successful
0808 02:18:05.619089 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":159,"threads_started":1}
I20260808 02:18:05.621377 30088 heartbeater.cc:347] Connected to a master server at 127.24.79.254:40249
I20260808 02:18:05.621644 30088 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:05.622263 30088 heartbeater.cc:511] Master 127.24.79.254:40249 requested a full tablet report, sending...
I20260808 02:18:05.623391 29413 ts_manager.cc:194] Registered new tserver with Master: 0204a1f185854701b35b2c6ee21583ac (127.24.79.195:41799)
I20260808 02:18:05.623992 29413 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.195@KRBTEST.COM'} at 127.24.79.195:54245
I20260808 02:18:05.634231 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 02:18:05 dist-test-slave-1vmg krb5kdc[29354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155485, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 02:18:05 dist-test-slave-1vmg krb5kdc[29354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155485, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:18:05.663328 29473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:05.653925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53410 (local address 127.24.79.254:40249)
0808 02:18:05.654094 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:05.654100 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:05.654202 (+ 102us) server_negotiation.cc:407] Connection header received
0808 02:18:05.654354 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:05.654358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:05.654406 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:05.654478 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:05.655294 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:05.655829 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:05.656680 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:05.656905 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:05.656998 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:05.659678 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:05.659682 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:05.659685 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:05.659719 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:05.661156 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:05.661737 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:05.661741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:05.661742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:05.661839 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:05.662111 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:05.662114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:05.662115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:05.662404 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:05.662515 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:05.663014 (+ 499us) server_negotiation.cc:299] Negotiation successful
0808 02:18:05.663194 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260808 02:18:05.663807 29413 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53410
I20260808 02:18:05.664760 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 29508
I20260808 02:18:05.687731 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 29703
I20260808 02:18:05.703893 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 29902
I20260808 02:18:05.719630 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 29374
2026-08-08T02:18:05Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2405 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:18:05 dist-test-slave-1vmg krb5kdc[30105](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:18:05 dist-test-slave-1vmg krb5kdc[30105](info): set up 2 sockets
Aug 08 02:18:05 dist-test-slave-1vmg krb5kdc[30105](info): commencing operation
krb5kdc: starting...
W20260808 02:18:06.885354 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:18:06 dist-test-slave-1vmg krb5kdc[30105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:18:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:18:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:18:07.026594 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:43191
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:34415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:43191
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:07.151682 30119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:07.152019 30119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:07.152086 30119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:07.156358 30119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:18:07.156451 30119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:07.156481 30119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:07.156504 30119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:18:07.156523 30119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:18:07.162352 30119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:34415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:43191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:43191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:07.163704 30119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:07.165102 30119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:07.175251 30127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:07.175302 30131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:07.175340 30128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:07.175428 30119 server_base.cc:1034] running on GCE node
I20260808 02:18:07.176357 30119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:07.177711 30119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:07.178941 30119 hybrid_clock.cc:648] HybridClock initialized: now 1786155487178891 us; error 44 us; skew 500 ppm
Aug 08 02:18:07 dist-test-slave-1vmg krb5kdc[30105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155487, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:07.182204 30119 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:18:07.183337 30119 webserver.cc:460] Webserver started at http://127.24.79.254:43999/ using document root <none> and password file <none>
I20260808 02:18:07.183946 30119 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:07.184021 30119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:07.184258 30119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:07.186539 30119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "c01fc8a32f8141c4ac7130590c0c6596"
format_stamp: "Formatted at 2026-08-08 02:18:07 on dist-test-slave-1vmg"
server_key: "9ac7aa30e8cfe208a863f9317c42d167"
server_key_iv: "6363e1100f306e5615e19850be1c2297"
server_key_version: "encryptionkey@0"
I20260808 02:18:07.187209 30119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "c01fc8a32f8141c4ac7130590c0c6596"
format_stamp: "Formatted at 2026-08-08 02:18:07 on dist-test-slave-1vmg"
server_key: "9ac7aa30e8cfe208a863f9317c42d167"
server_key_iv: "6363e1100f306e5615e19850be1c2297"
server_key_version: "encryptionkey@0"
I20260808 02:18:07.191550 30119 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:18:07.194278 30141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:07.196009 30119 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:07.196151 30119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "c01fc8a32f8141c4ac7130590c0c6596"
format_stamp: "Formatted at 2026-08-08 02:18:07 on dist-test-slave-1vmg"
server_key: "9ac7aa30e8cfe208a863f9317c42d167"
server_key_iv: "6363e1100f306e5615e19850be1c2297"
server_key_version: "encryptionkey@0"
I20260808 02:18:07.196305 30119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:07.233738 30119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:07.238020 30119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:07.238252 30119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:07.246361 30119 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:43191
I20260808 02:18:07.246395 30214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:43191 every 8 connection(s)
I20260808 02:18:07.247557 30119 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:18:07.250949 30215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:07.256490 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 30119
I20260808 02:18:07.256615 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:18:07.256877 24895 external_mini_cluster.cc:1428] Setting key b0ed801ac2e5c8228249d31b5668fb4d
I20260808 02:18:07.257452 30215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c01fc8a32f8141c4ac7130590c0c6596: Bootstrap starting.
I20260808 02:18:07.260440 30215 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c01fc8a32f8141c4ac7130590c0c6596: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:07.261471 30215 log.cc:824] T 00000000000000000000000000000000 P c01fc8a32f8141c4ac7130590c0c6596: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:07.264038 30215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c01fc8a32f8141c4ac7130590c0c6596: No bootstrap required, opened a new log
Aug 08 02:18:07 dist-test-slave-1vmg krb5kdc[30105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:18:07.268020 30215 raft_consensus.cc:348] T 00000000000000000000000000000000 P c01fc8a32f8141c4ac7130590c0c6596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01fc8a32f8141c4ac7130590c0c6596" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 43191 } }
I20260808 02:18:07.268311 30215 raft_consensus.cc:374] T 00000000000000000000000000000000 P c01fc8a32f8141c4ac7130590c0c6596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:07.268378 30215 raft_consensus.cc:729] T 00000000000000000000000000000000 P c01fc8a32f8141c4ac7130590c0c6596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01fc8a32f8141c4ac7130590c0c6596, State: Initialized, Role: FOLLOWER
I20260808 02:18:07.268944 30215 consensus_queue.cc:260] T 00000000000000000000000000000000 P c01fc8a32f8141c4ac7130590c0c6596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01fc8a32f8141c4ac7130590c0c6596" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 43191 } }
I20260808 02:18:07.269111 30215 raft_consensus.cc:388] T 00000000000000000000000000000000 P c01fc8a32f8141c4ac7130590c0c6596 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:18:07.269196 30215 raft_consensus.cc:482] T 00000000000000000000000000000000 P c01fc8a32f8141c4ac7130590c0c6596 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:18:07.269305 30215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c01fc8a32f8141c4ac7130590c0c6596 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:07.270644 30215 raft_consensus.cc:504] T 00000000000000000000000000000000 P c01fc8a32f8141c4ac7130590c0c6596 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01fc8a32f8141c4ac7130590c0c6596" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 43191 } }
I20260808 02:18:07.271118 30215 leader_election.cc:302] T 00000000000000000000000000000000 P c01fc8a32f8141c4ac7130590c0c6596 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c01fc8a32f8141c4ac7130590c0c6596; no voters:
I20260808 02:18:07.271528 30215 leader_election.cc:288] T 00000000000000000000000000000000 P c01fc8a32f8141c4ac7130590c0c6596 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:18:07.271647 30222 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c01fc8a32f8141c4ac7130590c0c6596 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:07.271903 30222 raft_consensus.cc:686] T 00000000000000000000000000000000 P c01fc8a32f8141c4ac7130590c0c6596 [term 1 LEADER]: Becoming Leader. State: Replica: c01fc8a32f8141c4ac7130590c0c6596, State: Running, Role: LEADER
I20260808 02:18:07.272331 30222 consensus_queue.cc:237] T 00000000000000000000000000000000 P c01fc8a32f8141c4ac7130590c0c6596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01fc8a32f8141c4ac7130590c0c6596" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 43191 } }
I20260808 02:18:07.273152 30215 sys_catalog.cc:564] T 00000000000000000000000000000000 P c01fc8a32f8141c4ac7130590c0c6596 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:18:07.273412 30218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:07.258598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53664 (local address 127.24.79.254:43191)
0808 02:18:07.259088 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:07.259109 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:07.259158 (+ 49us) server_negotiation.cc:407] Connection header received
0808 02:18:07.260089 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:07.260118 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:07.260607 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:07.261054 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:07.261899 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:07.263289 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:07.264247 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:07.264774 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:07.264960 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:07.267542 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:07.267548 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:07.267567 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:07.267605 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:07.269624 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:07.270244 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:07.270252 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:07.270261 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:07.270447 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:07.270815 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:07.270819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:07.270821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:07.271424 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:07.271656 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:07.272094 (+ 438us) server_negotiation.cc:299] Negotiation successful
0808 02:18:07.272480 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":123,"threads_started":1}
I20260808 02:18:07.273757 30222 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01fc8a32f8141c4ac7130590c0c6596 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c01fc8a32f8141c4ac7130590c0c6596. Latest consensus state: current_term: 1 leader_uuid: "c01fc8a32f8141c4ac7130590c0c6596" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01fc8a32f8141c4ac7130590c0c6596" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 43191 } } }
I20260808 02:18:07.273902 30222 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01fc8a32f8141c4ac7130590c0c6596 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:07.274130 30222 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01fc8a32f8141c4ac7130590c0c6596 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c01fc8a32f8141c4ac7130590c0c6596" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01fc8a32f8141c4ac7130590c0c6596" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 43191 } } }
I20260808 02:18:07.274408 30222 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01fc8a32f8141c4ac7130590c0c6596 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:07.274969 30229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:18:07.278736 30229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:18:07.285857 30229 catalog_manager.cc:1347] Generated new cluster ID: dafb352fdf42432f8cf223f336ca755b
I20260808 02:18:07.285971 30229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:18:07.295563 30229 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:18:07.297073 30229 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:18:07.302878 30229 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c01fc8a32f8141c4ac7130590c0c6596: Generated new TSK 0
I20260808 02:18:07.303817 30229 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.79.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
I20260808 02:18:07.342374 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:43191
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:34415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:07.467595 30255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:07.467926 30255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:07.467990 30255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:07.472244 30255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:07.472332 30255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:07.472419 30255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:18:07.478268 30255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:34415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:43191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:07.479619 30255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:07.480785 30255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:07.491662 30263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:07.491663 30268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:07.491720 30262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:07.491906 30255 server_base.cc:1034] running on GCE node
I20260808 02:18:07.492426 30255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:07.493191 30255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:07.494400 30255 hybrid_clock.cc:648] HybridClock initialized: now 1786155487494386 us; error 31 us; skew 500 ppm
Aug 08 02:18:07 dist-test-slave-1vmg krb5kdc[30105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155487, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:07.497664 30255 init.cc:377] Logged in from keytab as kudu/127.24.79.193@KRBTEST.COM (short username kudu)
I20260808 02:18:07.498694 30255 webserver.cc:460] Webserver started at http://127.24.79.193:38051/ using document root <none> and password file <none>
I20260808 02:18:07.499346 30255 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:07.499416 30255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:07.499603 30255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:07.502084 30255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "a16c01c43736458592c8bc6c580df556"
format_stamp: "Formatted at 2026-08-08 02:18:07 on dist-test-slave-1vmg"
server_key: "3293f9a84c8b79f7218d5a0fc2a955f8"
server_key_iv: "ab3850148955acae234167133a892bad"
server_key_version: "encryptionkey@0"
I20260808 02:18:07.502691 30255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "a16c01c43736458592c8bc6c580df556"
format_stamp: "Formatted at 2026-08-08 02:18:07 on dist-test-slave-1vmg"
server_key: "3293f9a84c8b79f7218d5a0fc2a955f8"
server_key_iv: "ab3850148955acae234167133a892bad"
server_key_version: "encryptionkey@0"
I20260808 02:18:07.507023 30255 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:18:07.509958 30279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:07.511562 30255 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:07.511713 30255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "a16c01c43736458592c8bc6c580df556"
format_stamp: "Formatted at 2026-08-08 02:18:07 on dist-test-slave-1vmg"
server_key: "3293f9a84c8b79f7218d5a0fc2a955f8"
server_key_iv: "ab3850148955acae234167133a892bad"
server_key_version: "encryptionkey@0"
I20260808 02:18:07.511829 30255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:07.534233 30255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:07.538112 30255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:07.538321 30255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:07.539034 30255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:07.540015 30255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:07.540068 30255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:07.540112 30255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:07.540128 30255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:07.549877 30255 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:36311
I20260808 02:18:07.549906 30443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:36311 every 8 connection(s)
I20260808 02:18:07.551008 30255 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:18:07.551625 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 30255
I20260808 02:18:07.551756 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:18:07.552031 24895 external_mini_cluster.cc:1428] Setting key 18b9d38266a153dd0ba77025e8837fd2
Aug 08 02:18:07 dist-test-slave-1vmg krb5kdc[30105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155487, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.79.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.194@KRBTEST.COM" created.
I20260808 02:18:07.568266 30218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:07.553303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:45565 (local address 127.24.79.254:43191)
0808 02:18:07.553461 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:07.553466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:07.554714 (+ 1248us) server_negotiation.cc:407] Connection header received
0808 02:18:07.556511 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:07.556515 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:07.556577 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:07.556676 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:07.558038 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:07.558643 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:07.560055 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:07.560301 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:07.560409 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:07.563691 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:07.563695 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:07.563697 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:07.563730 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:07.565408 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:07.566146 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:07.566150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:07.566201 (+ 51us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:07.566307 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:07.566812 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:07.566816 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:07.566817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:07.567126 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:07.567261 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:07.567907 (+ 646us) server_negotiation.cc:299] Negotiation successful
0808 02:18:07.568070 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:18:07.569227 30446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:07.553899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43191 (local address 127.24.79.193:45565)
0808 02:18:07.554509 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:07.554576 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:07.556206 (+ 1630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:07.556863 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:07.556870 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:07.557282 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:07.557841 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:07.557858 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:07.558810 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:07.558814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:07.559903 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:07.559917 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:07.560485 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:07.560488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:07.560688 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:07.561426 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:07.561447 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:07.563419 (+ 1972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:07.565587 (+ 2168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:07.565595 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:07.565616 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:07.565974 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:07.566444 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:07.566446 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:07.566448 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:07.566654 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:07.567261 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:07.567271 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:07.567611 (+ 340us) client_negotiation.cc:769] Sending connection context
0808 02:18:07.567813 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 02:18:07.568240 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":142,"threads_started":1}
I20260808 02:18:07.570869 30444 heartbeater.cc:347] Connected to a master server at 127.24.79.254:43191
I20260808 02:18:07.571192 30444 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:07.571813 30444 heartbeater.cc:511] Master 127.24.79.254:43191 requested a full tablet report, sending...
I20260808 02:18:07.573817 30158 ts_manager.cc:194] Registered new tserver with Master: a16c01c43736458592c8bc6c580df556 (127.24.79.193:36311)
I20260808 02:18:07.575188 30158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.193@KRBTEST.COM'} at 127.24.79.193:45565
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
I20260808 02:18:07.584545 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:43191
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:34415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:07.705924 30449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:07.706261 30449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:07.706308 30449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:07.711153 30449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:07.711258 30449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:07.711333 30449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:18:07.717482 30449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:34415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:43191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:07.718813 30449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:07.720090 30449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:07.731323 30458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:07.731323 30456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:07.731315 30461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:07.731472 30449 server_base.cc:1034] running on GCE node
I20260808 02:18:07.732177 30449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:07.732972 30449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:07.734185 30449 hybrid_clock.cc:648] HybridClock initialized: now 1786155487734160 us; error 43 us; skew 500 ppm
Aug 08 02:18:07 dist-test-slave-1vmg krb5kdc[30105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155487, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:07.737478 30449 init.cc:377] Logged in from keytab as kudu/127.24.79.194@KRBTEST.COM (short username kudu)
I20260808 02:18:07.738766 30449 webserver.cc:460] Webserver started at http://127.24.79.194:44709/ using document root <none> and password file <none>
I20260808 02:18:07.739446 30449 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:07.739521 30449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:07.739740 30449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:07.742141 30449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "67dfe3b8c744420fa6859722e4aecc8a"
format_stamp: "Formatted at 2026-08-08 02:18:07 on dist-test-slave-1vmg"
server_key: "16233b908057eaf334da8563eebed281"
server_key_iv: "d94d19dfbff3de04f9f3bd2354943733"
server_key_version: "encryptionkey@0"
I20260808 02:18:07.742762 30449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "67dfe3b8c744420fa6859722e4aecc8a"
format_stamp: "Formatted at 2026-08-08 02:18:07 on dist-test-slave-1vmg"
server_key: "16233b908057eaf334da8563eebed281"
server_key_iv: "d94d19dfbff3de04f9f3bd2354943733"
server_key_version: "encryptionkey@0"
I20260808 02:18:07.747280 30449 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:18:07.750103 30471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:07.751622 30449 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:07.751747 30449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "67dfe3b8c744420fa6859722e4aecc8a"
format_stamp: "Formatted at 2026-08-08 02:18:07 on dist-test-slave-1vmg"
server_key: "16233b908057eaf334da8563eebed281"
server_key_iv: "d94d19dfbff3de04f9f3bd2354943733"
server_key_version: "encryptionkey@0"
I20260808 02:18:07.751878 30449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:07.768456 30449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:07.772082 30449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:07.772305 30449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:07.773006 30449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:07.774022 30449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:07.774078 30449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:07.774125 30449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:07.774147 30449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:07.783895 30449 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:41917
I20260808 02:18:07.783960 30633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:41917 every 8 connection(s)
I20260808 02:18:07.784916 30449 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:18:07.793366 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 30449
I20260808 02:18:07.793494 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:18:07.793792 24895 external_mini_cluster.cc:1428] Setting key 3c0911baaa7dc0d91ef0af49c494f8ab
Aug 08 02:18:07 dist-test-slave-1vmg krb5kdc[30105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155487, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:18:07.801064 30218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:07.787221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:43415 (local address 127.24.79.254:43191)
0808 02:18:07.787412 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:07.787416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:07.788434 (+ 1018us) server_negotiation.cc:407] Connection header received
0808 02:18:07.789844 (+ 1410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:07.789847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:07.789914 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:07.789994 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:07.791433 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:07.791981 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:07.793300 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:07.793649 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:07.793769 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:07.796892 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:07.796896 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:07.796898 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:07.796927 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:07.798520 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:07.799205 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:07.799208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:07.799211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:07.799327 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:07.799695 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:07.799698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:07.799700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:07.799958 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:07.800057 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:07.800723 (+ 666us) server_negotiation.cc:299] Negotiation successful
0808 02:18:07.800878 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 02:18:07.802069 30636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:07.787610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43191 (local address 127.24.79.194:43415)
0808 02:18:07.788242 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:07.788296 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:07.789591 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:07.790221 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:07.790229 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:07.790641 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:07.791268 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:07.791285 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:07.792123 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:07.792126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:07.793109 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:07.793123 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:07.793855 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:07.793861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:07.794089 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:07.794804 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:07.794823 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:07.796685 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:07.798677 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:07.798684 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:07.798712 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:07.799076 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:07.799420 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:07.799423 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:07.799425 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:07.799590 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:07.800059 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:07.800065 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:07.800418 (+ 353us) client_negotiation.cc:769] Sending connection context
0808 02:18:07.800620 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 02:18:07.800953 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":141,"threads_started":1}
I20260808 02:18:07.803483 30634 heartbeater.cc:347] Connected to a master server at 127.24.79.254:43191
I20260808 02:18:07.803802 30634 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:07.804497 30634 heartbeater.cc:511] Master 127.24.79.254:43191 requested a full tablet report, sending...
I20260808 02:18:07.805954 30158 ts_manager.cc:194] Registered new tserver with Master: 67dfe3b8c744420fa6859722e4aecc8a (127.24.79.194:41917)
WARNING: no policy specified for kudu/127.24.79.195@KRBTEST.COM; defaulting to no policy
I20260808 02:18:07.806633 30158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.194@KRBTEST.COM'} at 127.24.79.194:43415
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
I20260808 02:18:07.827456 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:43191
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:34415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:07.951639 30639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:07.951952 30639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:07.952016 30639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:07.956305 30639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:07.956387 30639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:07.956475 30639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:18:07.962182 30639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:34415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:43191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:07.963595 30639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:07.964876 30639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:07.975442 30648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:07.975556 30649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:07.975613 30651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:07.976540 30639 server_base.cc:1034] running on GCE node
I20260808 02:18:07.977051 30639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:07.977866 30639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:07.979040 30639 hybrid_clock.cc:648] HybridClock initialized: now 1786155487979016 us; error 39 us; skew 500 ppm
Aug 08 02:18:07 dist-test-slave-1vmg krb5kdc[30105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155487, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:07.982355 30639 init.cc:377] Logged in from keytab as kudu/127.24.79.195@KRBTEST.COM (short username kudu)
I20260808 02:18:07.983451 30639 webserver.cc:460] Webserver started at http://127.24.79.195:39043/ using document root <none> and password file <none>
I20260808 02:18:07.984043 30639 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:07.984108 30639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:07.984320 30639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:07.986768 30639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "12ba4cef77d54d6f832791d9c0a47e90"
format_stamp: "Formatted at 2026-08-08 02:18:07 on dist-test-slave-1vmg"
server_key: "854b9b086a8360ac2169ba8301f20bea"
server_key_iv: "834e9c8e5272628ef75d5465c5b13e43"
server_key_version: "encryptionkey@0"
I20260808 02:18:07.987450 30639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "12ba4cef77d54d6f832791d9c0a47e90"
format_stamp: "Formatted at 2026-08-08 02:18:07 on dist-test-slave-1vmg"
server_key: "854b9b086a8360ac2169ba8301f20bea"
server_key_iv: "834e9c8e5272628ef75d5465c5b13e43"
server_key_version: "encryptionkey@0"
I20260808 02:18:07.991854 30639 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:18:07.994858 30663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:07.996637 30639 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:07.996797 30639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "12ba4cef77d54d6f832791d9c0a47e90"
format_stamp: "Formatted at 2026-08-08 02:18:07 on dist-test-slave-1vmg"
server_key: "854b9b086a8360ac2169ba8301f20bea"
server_key_iv: "834e9c8e5272628ef75d5465c5b13e43"
server_key_version: "encryptionkey@0"
I20260808 02:18:07.996940 30639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:08.019829 30639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:08.023622 30639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:08.023821 30639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:08.024514 30639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:08.025511 30639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:08.025563 30639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:08.025619 30639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:08.025650 30639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:08.035149 30639 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:36241
I20260808 02:18:08.035282 30818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:36241 every 8 connection(s)
I20260808 02:18:08.036454 30639 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:18:08.046679 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 30639
I20260808 02:18:08.046821 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:18:08.047143 24895 external_mini_cluster.cc:1428] Setting key af61b12240a94a860b4390a92bd821c0
Aug 08 02:18:08 dist-test-slave-1vmg krb5kdc[30105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155487, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:18:08.052997 30218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:08.038686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:44551 (local address 127.24.79.254:43191)
0808 02:18:08.038897 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:08.038902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:08.040060 (+ 1158us) server_negotiation.cc:407] Connection header received
0808 02:18:08.041686 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:08.041691 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:08.041753 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:08.041842 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:08.043366 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:08.043987 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:08.045376 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:08.045661 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:08.045764 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:08.048892 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:08.048896 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:08.048899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:08.048935 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:08.050446 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:08.051200 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:08.051203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:08.051204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:08.051321 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:08.051750 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:08.051753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:08.051754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:08.051954 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:08.052029 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:08.052744 (+ 715us) server_negotiation.cc:299] Negotiation successful
0808 02:18:08.052867 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260808 02:18:08.054029 30824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:08.039215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43191 (local address 127.24.79.195:44551)
0808 02:18:08.039841 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:08.039915 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:08.041380 (+ 1465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:08.042088 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:08.042097 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:08.042569 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:08.043154 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:08.043171 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:08.044190 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:08.044199 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:08.045204 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:08.045218 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:08.045860 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:08.045863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:08.046053 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:08.046781 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:08.046804 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:08.048715 (+ 1911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:08.050587 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:08.050594 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:08.050611 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:08.051083 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:08.051440 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:08.051443 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:08.051444 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:08.051652 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:08.052043 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:08.052049 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:08.052472 (+ 423us) client_negotiation.cc:769] Sending connection context
0808 02:18:08.052665 (+ 193us) client_negotiation.cc:240] Negotiation successful
0808 02:18:08.052994 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":165,"threads_started":1}
I20260808 02:18:08.055325 30821 heartbeater.cc:347] Connected to a master server at 127.24.79.254:43191
I20260808 02:18:08.055601 30821 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:08.056170 30821 heartbeater.cc:511] Master 127.24.79.254:43191 requested a full tablet report, sending...
I20260808 02:18:08.057464 30158 ts_manager.cc:194] Registered new tserver with Master: 12ba4cef77d54d6f832791d9c0a47e90 (127.24.79.195:36241)
I20260808 02:18:08.058079 30158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.195@KRBTEST.COM'} at 127.24.79.195:44551
I20260808 02:18:08.061923 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 02:18:08 dist-test-slave-1vmg krb5kdc[30105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 02:18:08 dist-test-slave-1vmg krb5kdc[30105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.193@KRBTEST.COM
I20260808 02:18:08.092967 30829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:08.082103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41966 (local address 127.24.79.193:36311)
0808 02:18:08.082465 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:08.082473 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:08.082499 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:18:08.082553 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:08.082559 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:08.082797 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:08.082961 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:08.083717 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:08.084545 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:08.085430 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:08.085849 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:08.085955 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:08.088818 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:08.088833 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:08.088851 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:08.088885 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:08.090677 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:08.091276 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:08.091283 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:08.091284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:08.091395 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:08.091771 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:08.091773 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:08.091774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:08.092100 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:08.092289 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:08.092644 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 02:18:08.092813 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":106,"threads_started":1}
Aug 08 02:18:08 dist-test-slave-1vmg krb5kdc[30105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:18:08.106172 30218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:08.096780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53680 (local address 127.24.79.254:43191)
0808 02:18:08.096949 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:08.096953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:08.096970 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:18:08.097118 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:08.097122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:08.097172 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:08.097243 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:08.097925 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:08.098473 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:08.099391 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:08.099677 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:08.099784 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:08.102657 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:08.102661 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:08.102663 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:08.102697 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:08.104320 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:08.104829 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:08.104833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:08.104835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:08.104931 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:08.105195 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:08.105198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:08.105199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:08.105493 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:08.105605 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:08.105902 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 02:18:08.106029 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260808 02:18:08.106700 30158 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53680
I20260808 02:18:08.107545 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 30255
I20260808 02:18:08.139989 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 30449
I20260808 02:18:08.178525 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 30639
I20260808 02:18:08.199390 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 30119
2026-08-08T02:18:08Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2479 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:18:08 dist-test-slave-1vmg krb5kdc[30839](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:18:08 dist-test-slave-1vmg krb5kdc[30839](info): set up 2 sockets
Aug 08 02:18:08 dist-test-slave-1vmg krb5kdc[30839](info): commencing operation
krb5kdc: starting...
W20260808 02:18:09.353088 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:18:09 dist-test-slave-1vmg krb5kdc[30839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:18:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:18:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:18:09.494766 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:46291
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:46291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:09.617198 30856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:09.617518 30856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:09.617563 30856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:09.621841 30856 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:18:09.621937 30856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:09.621958 30856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:09.621973 30856 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:18:09.621987 30856 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:18:09.627748 30856 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:46291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:46291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:09.628999 30856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:09.630116 30856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:09.639318 30862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:09.639416 30865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:09.639534 30861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:09.639981 30856 server_base.cc:1034] running on GCE node
I20260808 02:18:09.640669 30856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:09.641940 30856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:09.643103 30856 hybrid_clock.cc:648] HybridClock initialized: now 1786155489643081 us; error 44 us; skew 500 ppm
Aug 08 02:18:09 dist-test-slave-1vmg krb5kdc[30839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155489, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:09.646296 30856 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:18:09.647272 30856 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:09.647357 30856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:09.647586 30856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:09.649904 30856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "ebc3ae7b695846acad982754a6e8a30b"
format_stamp: "Formatted at 2026-08-08 02:18:09 on dist-test-slave-1vmg"
server_key: "fb766dc74b6ee6689fe4e25a91305da4"
server_key_iv: "3e2c3acd4654a441644772e376727e93"
server_key_version: "encryptionkey@0"
I20260808 02:18:09.650558 30856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "ebc3ae7b695846acad982754a6e8a30b"
format_stamp: "Formatted at 2026-08-08 02:18:09 on dist-test-slave-1vmg"
server_key: "fb766dc74b6ee6689fe4e25a91305da4"
server_key_iv: "3e2c3acd4654a441644772e376727e93"
server_key_version: "encryptionkey@0"
I20260808 02:18:09.655197 30856 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.004s
I20260808 02:18:09.658128 30877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:09.659536 30856 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:09.659684 30856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "ebc3ae7b695846acad982754a6e8a30b"
format_stamp: "Formatted at 2026-08-08 02:18:09 on dist-test-slave-1vmg"
server_key: "fb766dc74b6ee6689fe4e25a91305da4"
server_key_iv: "3e2c3acd4654a441644772e376727e93"
server_key_version: "encryptionkey@0"
I20260808 02:18:09.659827 30856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:09.672521 30856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:09.676198 30856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:09.676410 30856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:09.684103 30856 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:46291
I20260808 02:18:09.684410 30948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:46291 every 8 connection(s)
I20260808 02:18:09.684885 30856 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:18:09.688107 30950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:09.693856 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 30856
I20260808 02:18:09.694023 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:18:09.694324 24895 external_mini_cluster.cc:1428] Setting key d15c47ed6144cc42b5cec870bb1a778e
I20260808 02:18:09.694763 30950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b: Bootstrap starting.
I20260808 02:18:09.697597 30950 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:09.698590 30950 log.cc:824] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:09.701212 30950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b: No bootstrap required, opened a new log
Aug 08 02:18:09 dist-test-slave-1vmg krb5kdc[30839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:18:09.705937 30950 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebc3ae7b695846acad982754a6e8a30b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46291 } }
I20260808 02:18:09.706280 30950 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:09.706351 30950 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebc3ae7b695846acad982754a6e8a30b, State: Initialized, Role: FOLLOWER
I20260808 02:18:09.706939 30950 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebc3ae7b695846acad982754a6e8a30b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46291 } }
I20260808 02:18:09.707105 30950 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:18:09.707187 30950 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:18:09.707298 30950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:09.708335 30950 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebc3ae7b695846acad982754a6e8a30b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46291 } }
I20260808 02:18:09.708745 30950 leader_election.cc:302] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebc3ae7b695846acad982754a6e8a30b; no voters:
I20260808 02:18:09.709141 30950 leader_election.cc:288] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:18:09.709239 30958 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:09.709441 30958 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b [term 1 LEADER]: Becoming Leader. State: Replica: ebc3ae7b695846acad982754a6e8a30b, State: Running, Role: LEADER
I20260808 02:18:09.709806 30958 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebc3ae7b695846acad982754a6e8a30b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46291 } }
I20260808 02:18:09.710680 30950 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:18:09.711015 30954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:09.695959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43670 (local address 127.24.79.254:46291)
0808 02:18:09.696533 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:09.696555 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:09.696595 (+ 40us) server_negotiation.cc:407] Connection header received
0808 02:18:09.697386 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:09.697409 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:09.697815 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:09.698169 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:09.699232 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:09.700405 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:09.701561 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:09.702095 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:09.702233 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:09.705331 (+ 3098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:09.705337 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:09.705357 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:09.705394 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:09.707332 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:09.707851 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:09.707859 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:09.707866 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:09.708010 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:09.708306 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:09.708309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:09.708311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:09.708867 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:09.709083 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:09.709749 (+ 666us) server_negotiation.cc:299] Negotiation successful
0808 02:18:09.710083 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":195,"threads_started":1}
I20260808 02:18:09.712468 30958 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebc3ae7b695846acad982754a6e8a30b. Latest consensus state: current_term: 1 leader_uuid: "ebc3ae7b695846acad982754a6e8a30b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebc3ae7b695846acad982754a6e8a30b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46291 } } }
I20260808 02:18:09.712620 30958 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:09.713394 30961 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebc3ae7b695846acad982754a6e8a30b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebc3ae7b695846acad982754a6e8a30b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46291 } } }
I20260808 02:18:09.713517 30961 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b [sys.catalog]: This master's current role is: LEADER
W20260808 02:18:09.716099 30978 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:18:09.716194 30978 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:18:09.716300 30976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:18:09.717067 30976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:18:09.724006 30976 catalog_manager.cc:1347] Generated new cluster ID: 514dc23f4cfa42eeb6e1ae6af4b80c0a
I20260808 02:18:09.724109 30976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:18:09.734198 30976 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:18:09.735592 30976 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:18:09.748421 30976 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b: Generated new TSK 0
I20260808 02:18:09.749313 30976 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.79.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
I20260808 02:18:09.788523 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:46291
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:09.919978 30988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:09.920331 30988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:09.920384 30988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:09.924698 30988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:09.924790 30988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:09.924855 30988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:18:09.931062 30988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:46291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:09.932363 30988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:09.933562 30988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:09.945034 30996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:09.945096 31000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:09.945034 30998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:09.945461 30988 server_base.cc:1034] running on GCE node
I20260808 02:18:09.946108 30988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:09.946900 30988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:09.948161 30988 hybrid_clock.cc:648] HybridClock initialized: now 1786155489948140 us; error 38 us; skew 500 ppm
Aug 08 02:18:09 dist-test-slave-1vmg krb5kdc[30839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155489, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:09.951547 30988 init.cc:377] Logged in from keytab as kudu/127.24.79.193@KRBTEST.COM (short username kudu)
I20260808 02:18:09.952495 30988 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:09.952582 30988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:09.952837 30988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:09.955317 30988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "31a4e2cd161d4715b45f28762c74399a"
format_stamp: "Formatted at 2026-08-08 02:18:09 on dist-test-slave-1vmg"
server_key: "f5d36088c25bde9b826ce3ee3c2c8678"
server_key_iv: "a07a06a57bf4f49627eeedab719bda80"
server_key_version: "encryptionkey@0"
I20260808 02:18:09.955968 30988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "31a4e2cd161d4715b45f28762c74399a"
format_stamp: "Formatted at 2026-08-08 02:18:09 on dist-test-slave-1vmg"
server_key: "f5d36088c25bde9b826ce3ee3c2c8678"
server_key_iv: "a07a06a57bf4f49627eeedab719bda80"
server_key_version: "encryptionkey@0"
I20260808 02:18:09.961014 30988 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:18:09.964268 31006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:09.966122 30988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:18:09.966306 30988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "31a4e2cd161d4715b45f28762c74399a"
format_stamp: "Formatted at 2026-08-08 02:18:09 on dist-test-slave-1vmg"
server_key: "f5d36088c25bde9b826ce3ee3c2c8678"
server_key_iv: "a07a06a57bf4f49627eeedab719bda80"
server_key_version: "encryptionkey@0"
I20260808 02:18:09.966468 30988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:09.985311 30988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:09.989751 30988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:09.989957 30988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:09.990711 30988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:09.991799 30988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:09.991855 30988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:09.991901 30988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:09.991919 30988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:10.002652 30988 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:32999
I20260808 02:18:10.002748 31165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:32999 every 8 connection(s)
I20260808 02:18:10.003607 30988 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:18:10.008410 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 30988
I20260808 02:18:10.008543 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:18:10.008841 24895 external_mini_cluster.cc:1428] Setting key dff94aa2e871f4b1a846c9c41606ac52
Aug 08 02:18:10 dist-test-slave-1vmg krb5kdc[30839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155489, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.79.194@KRBTEST.COM; defaulting to no policy
I20260808 02:18:10.020509 30954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:10.005850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:44835 (local address 127.24.79.254:46291)
0808 02:18:10.005997 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:10.006001 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:10.007020 (+ 1019us) server_negotiation.cc:407] Connection header received
0808 02:18:10.008514 (+ 1494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:10.008518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:10.008597 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:10.008676 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:10.010145 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.010740 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.012249 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.012643 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.012759 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:10.015990 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:10.015994 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:10.015997 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:10.016049 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:10.017903 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:10.018579 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:10.018583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:10.018584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:10.018688 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:10.019156 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:10.019159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:10.019160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:10.019386 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:10.019483 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:10.020196 (+ 713us) server_negotiation.cc:299] Negotiation successful
0808 02:18:10.020362 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 02:18:10.021833 31171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:10.006210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:46291 (local address 127.24.79.193:44835)
0808 02:18:10.006794 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:10.006849 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:10.008238 (+ 1389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:10.008914 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:10.008925 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:10.009383 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:10.009965 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:10.009981 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.010900 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.010903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:10.012103 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:10.012117 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.012883 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.012888 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:10.013075 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:10.013784 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:10.013803 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:10.015690 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:10.018073 (+ 2383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:10.018082 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:10.018107 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:10.018440 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:10.018788 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:10.018792 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:10.018794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:10.019039 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:10.019507 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:10.019516 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:10.019868 (+ 352us) client_negotiation.cc:769] Sending connection context
0808 02:18:10.020078 (+ 210us) client_negotiation.cc:240] Negotiation successful
0808 02:18:10.020622 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":136,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.194@KRBTEST.COM" created.
I20260808 02:18:10.023705 31169 heartbeater.cc:347] Connected to a master server at 127.24.79.254:46291
I20260808 02:18:10.024058 31169 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:10.024659 31169 heartbeater.cc:511] Master 127.24.79.254:46291 requested a full tablet report, sending...
I20260808 02:18:10.026798 30901 ts_manager.cc:194] Registered new tserver with Master: 31a4e2cd161d4715b45f28762c74399a (127.24.79.193:32999)
I20260808 02:18:10.028223 30901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.193@KRBTEST.COM'} at 127.24.79.193:44835
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
I20260808 02:18:10.047573 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:46291
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:10.178432 31177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:10.178766 31177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:10.178815 31177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:10.183275 31177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:10.183373 31177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:10.183439 31177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:18:10.189250 31177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:46291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:10.190476 31177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:10.191628 31177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:10.202513 31187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:10.202513 31191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:10.202517 31189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:10.202584 31177 server_base.cc:1034] running on GCE node
I20260808 02:18:10.203284 31177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:10.204118 31177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:10.205307 31177 hybrid_clock.cc:648] HybridClock initialized: now 1786155490205289 us; error 42 us; skew 500 ppm
Aug 08 02:18:10 dist-test-slave-1vmg krb5kdc[30839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155490, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:10.208568 31177 init.cc:377] Logged in from keytab as kudu/127.24.79.194@KRBTEST.COM (short username kudu)
I20260808 02:18:10.209465 31177 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:10.209545 31177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:10.209774 31177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:10.212246 31177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "59fa931e099e40ebba01cc956607fc02"
format_stamp: "Formatted at 2026-08-08 02:18:10 on dist-test-slave-1vmg"
server_key: "c1ae84e19aaaeed72c22f34ef964a795"
server_key_iv: "49d53b84b517307cd2ded290c79046ff"
server_key_version: "encryptionkey@0"
I20260808 02:18:10.212842 31177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "59fa931e099e40ebba01cc956607fc02"
format_stamp: "Formatted at 2026-08-08 02:18:10 on dist-test-slave-1vmg"
server_key: "c1ae84e19aaaeed72c22f34ef964a795"
server_key_iv: "49d53b84b517307cd2ded290c79046ff"
server_key_version: "encryptionkey@0"
I20260808 02:18:10.217229 31177 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:18:10.219991 31197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:10.221437 31177 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 02:18:10.221604 31177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "59fa931e099e40ebba01cc956607fc02"
format_stamp: "Formatted at 2026-08-08 02:18:10 on dist-test-slave-1vmg"
server_key: "c1ae84e19aaaeed72c22f34ef964a795"
server_key_iv: "49d53b84b517307cd2ded290c79046ff"
server_key_version: "encryptionkey@0"
I20260808 02:18:10.221738 31177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:10.252199 31177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:10.255780 31177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:10.255985 31177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:10.256659 31177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:10.257611 31177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:10.257671 31177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:10.257728 31177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:10.257754 31177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:10.266863 31177 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:41499
I20260808 02:18:10.266904 31370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:41499 every 8 connection(s)
I20260808 02:18:10.267544 31177 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:18:10.277158 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 31177
I20260808 02:18:10.277278 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
Aug 08 02:18:10 dist-test-slave-1vmg krb5kdc[30839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155490, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:18:10.277568 24895 external_mini_cluster.cc:1428] Setting key eb84aecbb080c4fd0608d964d34e8dbf
I20260808 02:18:10.282059 30954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:10.269347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:57773 (local address 127.24.79.254:46291)
0808 02:18:10.269488 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:10.269492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:10.270648 (+ 1156us) server_negotiation.cc:407] Connection header received
0808 02:18:10.271600 (+ 952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:10.271603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:10.271666 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:10.271743 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:10.273010 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.273615 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.274934 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.275268 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.275369 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:10.278153 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:10.278156 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:10.278159 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:10.278187 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:10.279665 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:10.280319 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:10.280321 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:10.280322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:10.280404 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:10.280793 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:10.280795 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:10.280797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:10.280982 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:10.281054 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:10.281747 (+ 693us) server_negotiation.cc:299] Negotiation successful
0808 02:18:10.281892 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 02:18:10.282897 31373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:10.269776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:46291 (local address 127.24.79.194:57773)
0808 02:18:10.270486 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:10.270536 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:10.271394 (+ 858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:10.271925 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:10.271932 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:10.272344 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:10.272854 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:10.272869 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.273767 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.273771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:10.274764 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:10.274776 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.275452 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.275456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:10.275643 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:10.276290 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:10.276309 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:10.277909 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:10.279806 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:10.279813 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:10.279834 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:10.280219 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:10.280483 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:10.280486 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:10.280487 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:10.280703 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:10.281072 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:10.281079 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:10.281463 (+ 384us) client_negotiation.cc:769] Sending connection context
0808 02:18:10.281659 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 02:18:10.281958 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":118,"threads_started":1}
I20260808 02:18:10.284274 31371 heartbeater.cc:347] Connected to a master server at 127.24.79.254:46291
I20260808 02:18:10.284587 31371 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:10.285234 31371 heartbeater.cc:511] Master 127.24.79.254:46291 requested a full tablet report, sending...
I20260808 02:18:10.286722 30901 ts_manager.cc:194] Registered new tserver with Master: 59fa931e099e40ebba01cc956607fc02 (127.24.79.194:41499)
I20260808 02:18:10.287401 30901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.194@KRBTEST.COM'} at 127.24.79.194:57773
WARNING: no policy specified for kudu/127.24.79.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
I20260808 02:18:10.311415 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:46291
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:10.435768 31376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:10.436090 31376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:10.436160 31376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:10.440378 31376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:10.440466 31376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:10.440562 31376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:18:10.446343 31376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:46291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:10.447717 31376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:10.448868 31376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:10.459592 31381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:10.459780 31383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:10.459800 31376 server_base.cc:1034] running on GCE node
W20260808 02:18:10.459595 31385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:10.460508 31376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:10.461301 31376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:10.462584 31376 hybrid_clock.cc:648] HybridClock initialized: now 1786155490462563 us; error 35 us; skew 500 ppm
Aug 08 02:18:10 dist-test-slave-1vmg krb5kdc[30839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155490, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:10.466172 31376 init.cc:377] Logged in from keytab as kudu/127.24.79.195@KRBTEST.COM (short username kudu)
I20260808 02:18:10.467125 31376 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:10.467216 31376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:10.467460 31376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:10.469794 31376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "19eca6c58ae64a0782827ba0da99be7d"
format_stamp: "Formatted at 2026-08-08 02:18:10 on dist-test-slave-1vmg"
server_key: "c06f87c465a63a493130ad6af88b474f"
server_key_iv: "90a55ff9a7e7c3b10dc0fcea568ca776"
server_key_version: "encryptionkey@0"
I20260808 02:18:10.470436 31376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "19eca6c58ae64a0782827ba0da99be7d"
format_stamp: "Formatted at 2026-08-08 02:18:10 on dist-test-slave-1vmg"
server_key: "c06f87c465a63a493130ad6af88b474f"
server_key_iv: "90a55ff9a7e7c3b10dc0fcea568ca776"
server_key_version: "encryptionkey@0"
I20260808 02:18:10.475153 31376 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260808 02:18:10.478155 31396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:10.479688 31376 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:10.479887 31376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "19eca6c58ae64a0782827ba0da99be7d"
format_stamp: "Formatted at 2026-08-08 02:18:10 on dist-test-slave-1vmg"
server_key: "c06f87c465a63a493130ad6af88b474f"
server_key_iv: "90a55ff9a7e7c3b10dc0fcea568ca776"
server_key_version: "encryptionkey@0"
I20260808 02:18:10.480014 31376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:10.509243 31376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:10.513252 31376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:10.513460 31376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:10.514133 31376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:10.515108 31376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:10.515162 31376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:10.515219 31376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:10.515246 31376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:10.524699 31376 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:34293
I20260808 02:18:10.524762 31557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:34293 every 8 connection(s)
I20260808 02:18:10.525465 31376 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:18:10.530982 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 31376
I20260808 02:18:10.531128 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:18:10.531435 24895 external_mini_cluster.cc:1428] Setting key ea45adee4f8c10631b1a8740d2a16d65
Aug 08 02:18:10 dist-test-slave-1vmg krb5kdc[30839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155490, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:18:10.541814 30954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:10.527593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:59025 (local address 127.24.79.254:46291)
0808 02:18:10.527780 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:10.527785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:10.529204 (+ 1419us) server_negotiation.cc:407] Connection header received
0808 02:18:10.530154 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:10.530157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:10.530220 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:10.530311 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:10.532111 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.532738 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.534247 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.534490 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.534602 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:10.537524 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:10.537528 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:10.537530 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:10.537562 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:10.539162 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:10.539824 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:10.539827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:10.539828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:10.539927 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:10.540429 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:10.540433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:10.540434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:10.540696 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:10.540790 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:10.541504 (+ 714us) server_negotiation.cc:299] Negotiation successful
0808 02:18:10.541674 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 02:18:10.542675 31563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:10.528070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:46291 (local address 127.24.79.195:59025)
0808 02:18:10.528946 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:10.529000 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:10.529901 (+ 901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:10.530586 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:10.530595 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:10.531273 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:10.531915 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:10.531933 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.532900 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.532903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:10.534108 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:10.534121 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.534654 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.534657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:10.534824 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:10.535546 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:10.535566 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:10.537319 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:10.539339 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:10.539346 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:10.539365 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:10.539684 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:10.540054 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:10.540057 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:10.540058 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:10.540296 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:10.540793 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:10.540800 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:10.541128 (+ 328us) client_negotiation.cc:769] Sending connection context
0808 02:18:10.541366 (+ 238us) client_negotiation.cc:240] Negotiation successful
0808 02:18:10.541694 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":156,"threads_started":1}
I20260808 02:18:10.544040 31559 heartbeater.cc:347] Connected to a master server at 127.24.79.254:46291
I20260808 02:18:10.544345 31559 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:10.544890 31559 heartbeater.cc:511] Master 127.24.79.254:46291 requested a full tablet report, sending...
I20260808 02:18:10.546096 30901 ts_manager.cc:194] Registered new tserver with Master: 19eca6c58ae64a0782827ba0da99be7d (127.24.79.195:34293)
I20260808 02:18:10.546669 30901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.195@KRBTEST.COM'} at 127.24.79.195:59025
I20260808 02:18:10.558374 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:18:10.568616 30954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:10.560714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43684 (local address 127.24.79.254:46291)
0808 02:18:10.560856 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:10.560860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:10.561034 (+ 174us) server_negotiation.cc:407] Connection header received
0808 02:18:10.561177 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:10.561180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:10.561264 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:10.561330 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:10.562126 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.562703 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.563709 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.563959 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.564045 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:10.565470 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:10.565473 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:10.565475 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:10.565502 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:10.566744 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:10.567285 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:10.567288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:10.567290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:10.567374 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:10.567664 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:10.567668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:10.567670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:10.567931 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:10.568080 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:10.568368 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 02:18:10.568487 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 02:18:10.571914 30901 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:18:10.575269 30901 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:18:10.591010 31579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:10.585557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44988 (local address 127.24.79.194:41499)
0808 02:18:10.585919 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:10.585939 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:10.586288 (+ 349us) server_negotiation.cc:407] Connection header received
0808 02:18:10.586369 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:10.586373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:10.586550 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:10.586695 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:10.587263 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.588178 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.589594 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.590227 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.590334 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:10.590512 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:10.590696 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 02:18:10.590804 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":117,"threads_started":1}
I20260808 02:18:10.591008 31578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:10.585019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:41499 (local address 127.0.0.1:44988)
0808 02:18:10.585976 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:10.585991 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:10.586146 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:10.586908 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:10.586938 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:10.586970 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:10.587139 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:10.587146 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.588311 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.588314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:10.589430 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:10.589440 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.590358 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.590362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:10.590475 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:10.590515 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 02:18:10.590639 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 02:18:10.590804 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":886,"mutex_wait_us":31,"spinlock_wait_cycles":6528,"thread_start_us":138,"threads_started":1}
I20260808 02:18:10.591858 31576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:10.584868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:34293 (local address 127.0.0.1:59164)
0808 02:18:10.585919 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:10.585979 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:10.586145 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:10.586908 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:10.586938 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:10.586972 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:10.587133 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:10.587140 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.588985 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.588988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:10.590250 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:10.590262 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.591416 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.591420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:10.591516 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:10.591552 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 02:18:10.591628 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 02:18:10.591709 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856,"mutex_wait_us":8,"spinlock_wait_cycles":13184,"thread_start_us":89,"threads_started":1}
I20260808 02:18:10.592058 31575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:10.585252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:32999 (local address 127.0.0.1:48486)
0808 02:18:10.586053 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:10.586064 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:10.586145 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:10.587522 (+ 1377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:10.587525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:10.587554 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:10.587645 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:10.587651 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.589028 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.589031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:10.590256 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:10.590262 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.591634 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.591638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:10.591745 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:10.591770 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 02:18:10.591835 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 02:18:10.591913 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"mutex_wait_us":84,"thread_start_us":119,"threads_started":1}
I20260808 02:18:10.592615 31577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:10.584985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59164 (local address 127.24.79.195:34293)
0808 02:18:10.586443 (+ 1458us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:10.586455 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:10.586477 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:18:10.586555 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:10.586560 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:10.586751 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:10.586880 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:10.587631 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.588825 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.590400 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.591281 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.591392 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:10.591553 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:10.591689 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 02:18:10.591800 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1348,"thread_start_us":165,"threads_started":1}
I20260808 02:18:10.592643 31581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:10.585774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48486 (local address 127.24.79.193:32999)
0808 02:18:10.587154 (+ 1380us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:10.587161 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:10.587182 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:18:10.587254 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:10.587260 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:10.587397 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:10.587499 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:10.587773 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.588901 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.590401 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.591281 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.591392 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:10.592338 (+ 946us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:10.592397 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 02:18:10.592498 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1274,"thread_start_us":100,"threads_started":1}
I20260808 02:18:10.595026 31268 tablet_service.cc:1467] Processing CreateTablet for tablet c1ec6bdc860f45e5a3d0a7c10fd4c579 (DEFAULT_TABLE table=test-table [id=a951985b81174fa8b9d1c3726a67731d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:10.595641 31068 tablet_service.cc:1467] Processing CreateTablet for tablet c1ec6bdc860f45e5a3d0a7c10fd4c579 (DEFAULT_TABLE table=test-table [id=a951985b81174fa8b9d1c3726a67731d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:10.596148 31268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1ec6bdc860f45e5a3d0a7c10fd4c579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:10.596280 31463 tablet_service.cc:1467] Processing CreateTablet for tablet c1ec6bdc860f45e5a3d0a7c10fd4c579 (DEFAULT_TABLE table=test-table [id=a951985b81174fa8b9d1c3726a67731d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:10.596850 31068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1ec6bdc860f45e5a3d0a7c10fd4c579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:10.597347 31463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1ec6bdc860f45e5a3d0a7c10fd4c579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:10.603638 31582 tablet_bootstrap.cc:492] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 59fa931e099e40ebba01cc956607fc02: Bootstrap starting.
I20260808 02:18:10.605233 31584 tablet_bootstrap.cc:492] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d: Bootstrap starting.
I20260808 02:18:10.606422 31582 tablet_bootstrap.cc:654] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 59fa931e099e40ebba01cc956607fc02: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:10.607301 31582 log.cc:824] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 59fa931e099e40ebba01cc956607fc02: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:10.607366 31584 tablet_bootstrap.cc:654] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:10.607342 31583 tablet_bootstrap.cc:492] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 31a4e2cd161d4715b45f28762c74399a: Bootstrap starting.
I20260808 02:18:10.608798 31584 log.cc:824] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:10.609643 31582 tablet_bootstrap.cc:492] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 59fa931e099e40ebba01cc956607fc02: No bootstrap required, opened a new log
I20260808 02:18:10.609849 31583 tablet_bootstrap.cc:654] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 31a4e2cd161d4715b45f28762c74399a: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:10.609876 31582 ts_tablet_manager.cc:1397] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 59fa931e099e40ebba01cc956607fc02: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 02:18:10.610522 31583 log.cc:824] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 31a4e2cd161d4715b45f28762c74399a: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:10.611243 31584 tablet_bootstrap.cc:492] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d: No bootstrap required, opened a new log
I20260808 02:18:10.611443 31584 ts_tablet_manager.cc:1397] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260808 02:18:10.612926 31583 tablet_bootstrap.cc:492] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 31a4e2cd161d4715b45f28762c74399a: No bootstrap required, opened a new log
I20260808 02:18:10.613209 31583 ts_tablet_manager.cc:1397] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 31a4e2cd161d4715b45f28762c74399a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 02:18:10.614769 31584 raft_consensus.cc:348] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19eca6c58ae64a0782827ba0da99be7d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 34293 } } peers { permanent_uuid: "59fa931e099e40ebba01cc956607fc02" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41499 } } peers { permanent_uuid: "31a4e2cd161d4715b45f28762c74399a" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 32999 } }
I20260808 02:18:10.615075 31584 raft_consensus.cc:374] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:10.614833 31582 raft_consensus.cc:348] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 59fa931e099e40ebba01cc956607fc02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19eca6c58ae64a0782827ba0da99be7d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 34293 } } peers { permanent_uuid: "59fa931e099e40ebba01cc956607fc02" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41499 } } peers { permanent_uuid: "31a4e2cd161d4715b45f28762c74399a" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 32999 } }
I20260808 02:18:10.615135 31584 raft_consensus.cc:729] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19eca6c58ae64a0782827ba0da99be7d, State: Initialized, Role: FOLLOWER
I20260808 02:18:10.615180 31582 raft_consensus.cc:374] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 59fa931e099e40ebba01cc956607fc02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:10.615245 31582 raft_consensus.cc:729] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 59fa931e099e40ebba01cc956607fc02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59fa931e099e40ebba01cc956607fc02, State: Initialized, Role: FOLLOWER
I20260808 02:18:10.615790 31582 consensus_queue.cc:260] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 59fa931e099e40ebba01cc956607fc02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19eca6c58ae64a0782827ba0da99be7d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 34293 } } peers { permanent_uuid: "59fa931e099e40ebba01cc956607fc02" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41499 } } peers { permanent_uuid: "31a4e2cd161d4715b45f28762c74399a" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 32999 } }
I20260808 02:18:10.615856 31584 consensus_queue.cc:260] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19eca6c58ae64a0782827ba0da99be7d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 34293 } } peers { permanent_uuid: "59fa931e099e40ebba01cc956607fc02" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41499 } } peers { permanent_uuid: "31a4e2cd161d4715b45f28762c74399a" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 32999 } }
I20260808 02:18:10.616947 31582 ts_tablet_manager.cc:1428] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 59fa931e099e40ebba01cc956607fc02: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 02:18:10.616947 31584 ts_tablet_manager.cc:1428] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 02:18:10.617386 31559 heartbeater.cc:503] Master 127.24.79.254:46291 was elected leader, sending a full tablet report...
I20260808 02:18:10.617503 31371 heartbeater.cc:503] Master 127.24.79.254:46291 was elected leader, sending a full tablet report...
I20260808 02:18:10.617877 31583 raft_consensus.cc:348] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 31a4e2cd161d4715b45f28762c74399a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19eca6c58ae64a0782827ba0da99be7d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 34293 } } peers { permanent_uuid: "59fa931e099e40ebba01cc956607fc02" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41499 } } peers { permanent_uuid: "31a4e2cd161d4715b45f28762c74399a" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 32999 } }
I20260808 02:18:10.618183 31583 raft_consensus.cc:374] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 31a4e2cd161d4715b45f28762c74399a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:10.618248 31583 raft_consensus.cc:729] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 31a4e2cd161d4715b45f28762c74399a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31a4e2cd161d4715b45f28762c74399a, State: Initialized, Role: FOLLOWER
I20260808 02:18:10.618719 31583 consensus_queue.cc:260] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 31a4e2cd161d4715b45f28762c74399a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19eca6c58ae64a0782827ba0da99be7d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 34293 } } peers { permanent_uuid: "59fa931e099e40ebba01cc956607fc02" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41499 } } peers { permanent_uuid: "31a4e2cd161d4715b45f28762c74399a" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 32999 } }
I20260808 02:18:10.619447 31169 heartbeater.cc:503] Master 127.24.79.254:46291 was elected leader, sending a full tablet report...
I20260808 02:18:10.619864 31583 ts_tablet_manager.cc:1428] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 31a4e2cd161d4715b45f28762c74399a: Time spent starting tablet: real 0.007s user 0.000s sys 0.006s
I20260808 02:18:10.704241 31590 raft_consensus.cc:482] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:18:10.704506 31590 raft_consensus.cc:504] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19eca6c58ae64a0782827ba0da99be7d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 34293 } } peers { permanent_uuid: "59fa931e099e40ebba01cc956607fc02" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41499 } } peers { permanent_uuid: "31a4e2cd161d4715b45f28762c74399a" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 32999 } }
I20260808 02:18:10.705809 31590 leader_election.cc:288] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59fa931e099e40ebba01cc956607fc02 (127.24.79.194:41499), 31a4e2cd161d4715b45f28762c74399a (127.24.79.193:32999)
I20260808 02:18:10.710328 31581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:10.706312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:36063 (local address 127.24.79.193:32999)
0808 02:18:10.706445 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:10.706451 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:10.706473 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:18:10.706591 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:10.706594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:10.706651 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:10.706730 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:10.707090 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.707881 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.709229 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.709730 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.709830 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:10.709880 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:10.710108 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 02:18:10.710196 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 02:18:10.710345 31579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:10.706332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:45003 (local address 127.24.79.194:41499)
0808 02:18:10.706460 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:10.706465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:10.706851 (+ 386us) server_negotiation.cc:407] Connection header received
0808 02:18:10.706993 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:10.706995 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:10.707040 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:10.707186 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:10.707419 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.708107 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.709353 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.709807 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.709928 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:10.709981 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:10.710163 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 02:18:10.710242 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:18:10.710320 31563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:10.706177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:32999 (local address 127.24.79.195:36063)
0808 02:18:10.706344 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:10.706360 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:10.706474 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:10.706787 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:10.706791 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:10.706812 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:10.706972 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:10.706979 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.708011 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.708014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:10.709099 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:10.709110 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.709861 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.709865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:10.709971 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:10.709995 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 02:18:10.710063 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 02:18:10.710135 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"mutex_wait_us":64}
I20260808 02:18:10.710354 31592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:10.706186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:41499 (local address 127.24.79.195:45003)
0808 02:18:10.706742 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:10.706755 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:10.706866 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:10.707201 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:10.707203 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:10.707220 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:10.707319 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:10.707323 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.708220 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.708223 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:10.709232 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:10.709244 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.709940 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.709944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:10.710025 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:10.710048 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:18:10.710126 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 02:18:10.710194 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"mutex_wait_us":128,"spinlock_wait_cycles":768,"thread_start_us":151,"threads_started":1}
I20260808 02:18:10.711120 31099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ec6bdc860f45e5a3d0a7c10fd4c579" candidate_uuid: "19eca6c58ae64a0782827ba0da99be7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31a4e2cd161d4715b45f28762c74399a" is_pre_election: true
I20260808 02:18:10.711179 31290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ec6bdc860f45e5a3d0a7c10fd4c579" candidate_uuid: "19eca6c58ae64a0782827ba0da99be7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59fa931e099e40ebba01cc956607fc02" is_pre_election: true
I20260808 02:18:10.711481 31290 raft_consensus.cc:2454] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 59fa931e099e40ebba01cc956607fc02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19eca6c58ae64a0782827ba0da99be7d in term 0.
I20260808 02:18:10.711481 31099 raft_consensus.cc:2454] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 31a4e2cd161d4715b45f28762c74399a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19eca6c58ae64a0782827ba0da99be7d in term 0.
I20260808 02:18:10.712013 31399 leader_election.cc:302] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 19eca6c58ae64a0782827ba0da99be7d, 31a4e2cd161d4715b45f28762c74399a, 59fa931e099e40ebba01cc956607fc02; no voters:
I20260808 02:18:10.712330 31590 raft_consensus.cc:2781] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:18:10.712412 31590 raft_consensus.cc:482] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:18:10.712443 31590 raft_consensus.cc:3037] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:10.713634 31590 raft_consensus.cc:504] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19eca6c58ae64a0782827ba0da99be7d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 34293 } } peers { permanent_uuid: "59fa931e099e40ebba01cc956607fc02" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41499 } } peers { permanent_uuid: "31a4e2cd161d4715b45f28762c74399a" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 32999 } }
I20260808 02:18:10.714020 31590 leader_election.cc:288] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d [CANDIDATE]: Term 1 election: Requested vote from peers 59fa931e099e40ebba01cc956607fc02 (127.24.79.194:41499), 31a4e2cd161d4715b45f28762c74399a (127.24.79.193:32999)
I20260808 02:18:10.714540 31290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ec6bdc860f45e5a3d0a7c10fd4c579" candidate_uuid: "19eca6c58ae64a0782827ba0da99be7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59fa931e099e40ebba01cc956607fc02"
I20260808 02:18:10.714540 31099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ec6bdc860f45e5a3d0a7c10fd4c579" candidate_uuid: "19eca6c58ae64a0782827ba0da99be7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31a4e2cd161d4715b45f28762c74399a"
I20260808 02:18:10.714691 31290 raft_consensus.cc:3037] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 59fa931e099e40ebba01cc956607fc02 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:10.714697 31099 raft_consensus.cc:3037] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 31a4e2cd161d4715b45f28762c74399a [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:10.716156 31099 raft_consensus.cc:2454] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 31a4e2cd161d4715b45f28762c74399a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19eca6c58ae64a0782827ba0da99be7d in term 1.
I20260808 02:18:10.716166 31290 raft_consensus.cc:2454] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 59fa931e099e40ebba01cc956607fc02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19eca6c58ae64a0782827ba0da99be7d in term 1.
I20260808 02:18:10.716548 31399 leader_election.cc:302] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19eca6c58ae64a0782827ba0da99be7d, 31a4e2cd161d4715b45f28762c74399a; no voters:
I20260808 02:18:10.716743 31590 raft_consensus.cc:2781] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:10.717005 31590 raft_consensus.cc:686] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d [term 1 LEADER]: Becoming Leader. State: Replica: 19eca6c58ae64a0782827ba0da99be7d, State: Running, Role: LEADER
I20260808 02:18:10.717365 31590 consensus_queue.cc:237] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19eca6c58ae64a0782827ba0da99be7d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 34293 } } peers { permanent_uuid: "59fa931e099e40ebba01cc956607fc02" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41499 } } peers { permanent_uuid: "31a4e2cd161d4715b45f28762c74399a" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 32999 } }
I20260808 02:18:10.720873 30901 catalog_manager.cc:5495] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 19eca6c58ae64a0782827ba0da99be7d (127.24.79.195). New cstate: current_term: 1 leader_uuid: "19eca6c58ae64a0782827ba0da99be7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19eca6c58ae64a0782827ba0da99be7d" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 34293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59fa931e099e40ebba01cc956607fc02" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31a4e2cd161d4715b45f28762c74399a" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 32999 } health_report { overall_health: UNKNOWN } } }
I20260808 02:18:10.749416 31577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:10.745355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59178 (local address 127.24.79.195:34293)
0808 02:18:10.745503 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:10.745507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:10.745522 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:18:10.745642 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:10.745644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:10.745701 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:10.745798 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:18:10.746168 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.746708 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.747711 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:10.747943 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:10.748042 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:10.748457 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:18:10.748926 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:18:10.749023 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:10.749157 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 02:18:10.749219 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 02:18:10.755164 31170 tablet.cc:2369] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 31a4e2cd161d4715b45f28762c74399a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:18:10.756584 31099 raft_consensus.cc:1261] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 31a4e2cd161d4715b45f28762c74399a [term 1 FOLLOWER]: Refusing update from remote peer 19eca6c58ae64a0782827ba0da99be7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:18:10.756582 31290 raft_consensus.cc:1261] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 59fa931e099e40ebba01cc956607fc02 [term 1 FOLLOWER]: Refusing update from remote peer 19eca6c58ae64a0782827ba0da99be7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:18:10.757453 31590 consensus_queue.cc:1035] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "59fa931e099e40ebba01cc956607fc02" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:18:10.757704 31596 consensus_queue.cc:1035] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "31a4e2cd161d4715b45f28762c74399a" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 32999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:18:10.767025 31600 mvcc.cc:205] Tried to move back new op lower bound from 7316092890129494016 to 7316092889981149184. Current Snapshot: MvccSnapshot[applied={T|T < 7316092890129494016}]
I20260808 02:18:10.777904 30901 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:18:10.778095 30901 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:18:10.780747 30901 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ebc3ae7b695846acad982754a6e8a30b: Sending DeleteTablet for 3 replicas of tablet c1ec6bdc860f45e5a3d0a7c10fd4c579
I20260808 02:18:10.781697 31463 tablet_service.cc:1514] Processing DeleteTablet for tablet c1ec6bdc860f45e5a3d0a7c10fd4c579 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:10 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:59164
I20260808 02:18:10.781757 31268 tablet_service.cc:1514] Processing DeleteTablet for tablet c1ec6bdc860f45e5a3d0a7c10fd4c579 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:10 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:44988
I20260808 02:18:10.781991 31068 tablet_service.cc:1514] Processing DeleteTablet for tablet c1ec6bdc860f45e5a3d0a7c10fd4c579 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:10 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:48486
I20260808 02:18:10.782323 31615 tablet_replica.cc:330] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d: stopping tablet replica
I20260808 02:18:10.782666 31615 raft_consensus.cc:2229] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:18:10.782958 31616 tablet_replica.cc:330] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 59fa931e099e40ebba01cc956607fc02: stopping tablet replica
I20260808 02:18:10.782991 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 30988
I20260808 02:18:10.791600 31615 raft_consensus.cc:2258] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:18:10.792065 31616 raft_consensus.cc:2229] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 59fa931e099e40ebba01cc956607fc02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:18:10.792610 31616 raft_consensus.cc:2258] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 59fa931e099e40ebba01cc956607fc02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:18:10.794627 31615 ts_tablet_manager.cc:1905] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:18:10.797170 31616 ts_tablet_manager.cc:1905] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 59fa931e099e40ebba01cc956607fc02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:18:10.797839 31615 ts_tablet_manager.cc:1918] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:18:10.797962 31615 log.cc:1196] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/c1ec6bdc860f45e5a3d0a7c10fd4c579
I20260808 02:18:10.798310 31615 ts_tablet_manager.cc:1939] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 19eca6c58ae64a0782827ba0da99be7d: Deleting consensus metadata
I20260808 02:18:10.799883 30879 catalog_manager.cc:4841] TS 19eca6c58ae64a0782827ba0da99be7d (127.24.79.195:34293): tablet c1ec6bdc860f45e5a3d0a7c10fd4c579 (table test-table [id=a951985b81174fa8b9d1c3726a67731d]) successfully deleted
I20260808 02:18:10.800820 31616 ts_tablet_manager.cc:1918] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 59fa931e099e40ebba01cc956607fc02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:18:10.800936 31616 log.cc:1196] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 59fa931e099e40ebba01cc956607fc02: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/c1ec6bdc860f45e5a3d0a7c10fd4c579
I20260808 02:18:10.801208 31616 ts_tablet_manager.cc:1939] T c1ec6bdc860f45e5a3d0a7c10fd4c579 P 59fa931e099e40ebba01cc956607fc02: Deleting consensus metadata
W20260808 02:18:10.802191 30881 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.79.193:32999 (error 108)
I20260808 02:18:10.802908 30880 catalog_manager.cc:4841] TS 59fa931e099e40ebba01cc956607fc02 (127.24.79.194:41499): tablet c1ec6bdc860f45e5a3d0a7c10fd4c579 (table test-table [id=a951985b81174fa8b9d1c3726a67731d]) successfully deleted
W20260808 02:18:10.803413 31575 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:10.803102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:32999 (local address 127.0.0.1:48498)
0808 02:18:10.803239 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:10.803252 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:10.803321 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.193:32999: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 02:18:10.803601 30881 catalog_manager.cc:4584] TS 31a4e2cd161d4715b45f28762c74399a (127.24.79.193:32999): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c1ec6bdc860f45e5a3d0a7c10fd4c579: Network error: Client connection negotiation failed: client connection to 127.24.79.193:32999: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260808 02:18:10.803691 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 31177
I20260808 02:18:10.819367 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 31376
I20260808 02:18:10.837306 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 30856
2026-08-08T02:18:10Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2636 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T02:18:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:18:10Z Disabled control of system clock
I20260808 02:18:10.878290 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:45977
--webserver_interface=127.24.79.254
--webserver_port=0
--builtin_ntp_servers=127.24.79.212:40341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:45977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 02:18:10.999536 31626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:10.999894 31626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:10.999939 31626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:11.004142 31626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:18:11.004280 31626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:11.004297 31626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:11.004314 31626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:18:11.004329 31626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:18:11.009933 31626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:45977
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:45977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:11.011307 31626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:11.012853 31626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:11.022253 31634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:11.022332 31636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:11.022266 31632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:11.022789 31626 server_base.cc:1034] running on GCE node
I20260808 02:18:11.023495 31626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:11.024686 31626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:11.025906 31626 hybrid_clock.cc:648] HybridClock initialized: now 1786155491025879 us; error 43 us; skew 500 ppm
I20260808 02:18:11.031407 31626 webserver.cc:460] Webserver started at http://127.24.79.254:37159/ using document root <none> and password file <none>
I20260808 02:18:11.032114 31626 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:11.032195 31626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:11.032428 31626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:11.034825 31626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "0eeefcf85ff04d86a7b253d41c28b562"
format_stamp: "Formatted at 2026-08-08 02:18:11 on dist-test-slave-1vmg"
server_key: "9e0c07aed8f35ed9670b50c1bfdefdbe"
server_key_iv: "7638785fe242b72e13cf7d8afd172d2c"
server_key_version: "encryptionkey@0"
I20260808 02:18:11.035529 31626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "0eeefcf85ff04d86a7b253d41c28b562"
format_stamp: "Formatted at 2026-08-08 02:18:11 on dist-test-slave-1vmg"
server_key: "9e0c07aed8f35ed9670b50c1bfdefdbe"
server_key_iv: "7638785fe242b72e13cf7d8afd172d2c"
server_key_version: "encryptionkey@0"
I20260808 02:18:11.039750 31626 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:18:11.042526 31647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:11.044024 31626 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:18:11.044163 31626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "0eeefcf85ff04d86a7b253d41c28b562"
format_stamp: "Formatted at 2026-08-08 02:18:11 on dist-test-slave-1vmg"
server_key: "9e0c07aed8f35ed9670b50c1bfdefdbe"
server_key_iv: "7638785fe242b72e13cf7d8afd172d2c"
server_key_version: "encryptionkey@0"
I20260808 02:18:11.044286 31626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:11.049489 31626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:11.050207 31626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:11.050400 31626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:11.057857 31626 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:45977
I20260808 02:18:11.057879 31722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:45977 every 8 connection(s)
I20260808 02:18:11.058954 31626 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:18:11.062647 31723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:11.065624 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 31626
I20260808 02:18:11.065737 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:18:11.066009 24895 external_mini_cluster.cc:1428] Setting key b4262d84f2d974f34d217aeb95f4d794
I20260808 02:18:11.069185 31723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0eeefcf85ff04d86a7b253d41c28b562: Bootstrap starting.
I20260808 02:18:11.071805 31723 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0eeefcf85ff04d86a7b253d41c28b562: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:11.072250 31728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:11.067618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38964 (local address 127.24.79.254:45977)
0808 02:18:11.068391 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:11.068410 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:11.068451 (+ 41us) server_negotiation.cc:407] Connection header received
0808 02:18:11.069125 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:11.069143 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:11.069554 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:11.069916 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:11.070776 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:11.070781 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:11.070794 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:18:11.070807 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:11.070873 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:18:11.070890 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:11.071086 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:11.071164 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 02:18:11.071443 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":151,"threads_started":1}
I20260808 02:18:11.072705 31723 log.cc:824] T 00000000000000000000000000000000 P 0eeefcf85ff04d86a7b253d41c28b562: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:11.075186 31723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0eeefcf85ff04d86a7b253d41c28b562: No bootstrap required, opened a new log
I20260808 02:18:11.078989 31723 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0eeefcf85ff04d86a7b253d41c28b562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eeefcf85ff04d86a7b253d41c28b562" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45977 } }
I20260808 02:18:11.079278 31723 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0eeefcf85ff04d86a7b253d41c28b562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:11.079324 31723 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0eeefcf85ff04d86a7b253d41c28b562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0eeefcf85ff04d86a7b253d41c28b562, State: Initialized, Role: FOLLOWER
I20260808 02:18:11.079725 31723 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0eeefcf85ff04d86a7b253d41c28b562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eeefcf85ff04d86a7b253d41c28b562" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45977 } }
I20260808 02:18:11.079865 31723 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0eeefcf85ff04d86a7b253d41c28b562 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:18:11.079916 31723 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0eeefcf85ff04d86a7b253d41c28b562 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:18:11.079995 31723 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0eeefcf85ff04d86a7b253d41c28b562 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:11.081020 31723 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0eeefcf85ff04d86a7b253d41c28b562 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eeefcf85ff04d86a7b253d41c28b562" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45977 } }
I20260808 02:18:11.081393 31723 leader_election.cc:302] T 00000000000000000000000000000000 P 0eeefcf85ff04d86a7b253d41c28b562 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0eeefcf85ff04d86a7b253d41c28b562; no voters:
I20260808 02:18:11.081764 31723 leader_election.cc:288] T 00000000000000000000000000000000 P 0eeefcf85ff04d86a7b253d41c28b562 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:18:11.081831 31733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0eeefcf85ff04d86a7b253d41c28b562 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:11.082041 31733 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0eeefcf85ff04d86a7b253d41c28b562 [term 1 LEADER]: Becoming Leader. State: Replica: 0eeefcf85ff04d86a7b253d41c28b562, State: Running, Role: LEADER
I20260808 02:18:11.082371 31733 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0eeefcf85ff04d86a7b253d41c28b562 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eeefcf85ff04d86a7b253d41c28b562" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45977 } }
I20260808 02:18:11.083269 31723 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0eeefcf85ff04d86a7b253d41c28b562 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:18:11.084342 31734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0eeefcf85ff04d86a7b253d41c28b562 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0eeefcf85ff04d86a7b253d41c28b562" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eeefcf85ff04d86a7b253d41c28b562" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45977 } } }
I20260808 02:18:11.084488 31734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0eeefcf85ff04d86a7b253d41c28b562 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:11.084770 31735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0eeefcf85ff04d86a7b253d41c28b562 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0eeefcf85ff04d86a7b253d41c28b562. Latest consensus state: current_term: 1 leader_uuid: "0eeefcf85ff04d86a7b253d41c28b562" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eeefcf85ff04d86a7b253d41c28b562" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45977 } } }
I20260808 02:18:11.084863 31735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0eeefcf85ff04d86a7b253d41c28b562 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:11.084997 31742 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:18:11.088430 31742 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:18:11.094414 31742 catalog_manager.cc:1347] Generated new cluster ID: d174e7ab677840c081cfb675c50372a3
I20260808 02:18:11.094509 31742 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:18:11.129869 31742 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:18:11.130317 31742 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:18:11.137686 31742 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0eeefcf85ff04d86a7b253d41c28b562: Generated new TSK 0
I20260808 02:18:11.138655 31742 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:18:11.145046 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:45977
--builtin_ntp_servers=127.24.79.212:40341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 02:18:11.272744 31764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:11.273056 31764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:11.273098 31764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:11.277277 31764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:11.277371 31764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:11.277432 31764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:18:11.282572 31764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:45977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:11.283939 31764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:11.285564 31764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:11.296229 31771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:11.296216 31775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:11.296335 31772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:11.296485 31764 server_base.cc:1034] running on GCE node
I20260808 02:18:11.296986 31764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:11.297726 31764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:11.298967 31764 hybrid_clock.cc:648] HybridClock initialized: now 1786155491298940 us; error 33 us; skew 500 ppm
I20260808 02:18:11.301208 31764 webserver.cc:460] Webserver started at http://127.24.79.193:46097/ using document root <none> and password file <none>
I20260808 02:18:11.301867 31764 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:11.301937 31764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:11.302214 31764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:11.304633 31764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "b94f77d994be40569fa84a56d9ef1324"
format_stamp: "Formatted at 2026-08-08 02:18:11 on dist-test-slave-1vmg"
server_key: "ea4799771cc9d17bd72453904d54230c"
server_key_iv: "bdd1d8050deb4ad7aa2950b18701895c"
server_key_version: "encryptionkey@0"
I20260808 02:18:11.305254 31764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "b94f77d994be40569fa84a56d9ef1324"
format_stamp: "Formatted at 2026-08-08 02:18:11 on dist-test-slave-1vmg"
server_key: "ea4799771cc9d17bd72453904d54230c"
server_key_iv: "bdd1d8050deb4ad7aa2950b18701895c"
server_key_version: "encryptionkey@0"
I20260808 02:18:11.309408 31764 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:18:11.312341 31781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:11.313786 31764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:18:11.313944 31764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "b94f77d994be40569fa84a56d9ef1324"
format_stamp: "Formatted at 2026-08-08 02:18:11 on dist-test-slave-1vmg"
server_key: "ea4799771cc9d17bd72453904d54230c"
server_key_iv: "bdd1d8050deb4ad7aa2950b18701895c"
server_key_version: "encryptionkey@0"
I20260808 02:18:11.314080 31764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:11.319530 31764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:11.320318 31764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:11.320506 31764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:11.321136 31764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:11.322196 31764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:11.322247 31764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:11.322305 31764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:11.322331 31764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:11.334246 31764 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:46389
I20260808 02:18:11.334295 31951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:46389 every 8 connection(s)
I20260808 02:18:11.335664 31764 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:18:11.343500 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 31764
I20260808 02:18:11.343638 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:18:11.343983 24895 external_mini_cluster.cc:1428] Setting key c06db35d36e3fb51fd0e79ba677e0926
I20260808 02:18:11.346189 31728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:11.337945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:59665 (local address 127.24.79.254:45977)
0808 02:18:11.338074 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:11.338079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:11.341509 (+ 3430us) server_negotiation.cc:407] Connection header received
0808 02:18:11.343550 (+ 2041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:11.343555 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:11.343615 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:11.343723 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:11.345181 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:11.345185 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:11.345188 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:18:11.345202 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:11.345229 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:18:11.345239 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:11.345330 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:11.345995 (+ 665us) server_negotiation.cc:299] Negotiation successful
0808 02:18:11.346048 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 02:18:11.346362 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:45977
--builtin_ntp_servers=127.24.79.212:40341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 02:18:11.347631 31954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:11.338560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45977 (local address 127.24.79.193:59665)
0808 02:18:11.341278 (+ 2718us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:11.341355 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:11.343230 (+ 1875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:11.343968 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:11.343980 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:11.344132 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:11.344874 (+ 742us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:18:11.344876 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:11.344942 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:18:11.345030 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:11.345809 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:11.345822 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:11.345829 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 02:18:11.346046 (+ 217us) client_negotiation.cc:240] Negotiation successful
0808 02:18:11.346380 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2079,"thread_start_us":160,"threads_started":1}
I20260808 02:18:11.351171 31952 heartbeater.cc:347] Connected to a master server at 127.24.79.254:45977
I20260808 02:18:11.351483 31952 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:11.352051 31952 heartbeater.cc:511] Master 127.24.79.254:45977 requested a full tablet report, sending...
I20260808 02:18:11.356639 31674 ts_manager.cc:194] Registered new tserver with Master: b94f77d994be40569fa84a56d9ef1324 (127.24.79.193:46389)
W20260808 02:18:11.472189 31955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:11.472512 31955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:11.472555 31955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:11.476770 31955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:11.476847 31955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:11.476909 31955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:18:11.482015 31955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:45977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:11.483402 31955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:11.485126 31955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:11.496135 31964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:11.496243 31963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:11.496251 31955 server_base.cc:1034] running on GCE node
W20260808 02:18:11.496135 31970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:11.497117 31955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:11.497895 31955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:11.499078 31955 hybrid_clock.cc:648] HybridClock initialized: now 1786155491499055 us; error 40 us; skew 500 ppm
I20260808 02:18:11.501356 31955 webserver.cc:460] Webserver started at http://127.24.79.194:37443/ using document root <none> and password file <none>
I20260808 02:18:11.502048 31955 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:11.502132 31955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:11.502357 31955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:11.504812 31955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "9058b120d2f948fabde6d2ccf5c457cb"
format_stamp: "Formatted at 2026-08-08 02:18:11 on dist-test-slave-1vmg"
server_key: "27dca86b495c999631e4fd754eeb47ca"
server_key_iv: "4b29e04e03bab181e6629fa20d29ebba"
server_key_version: "encryptionkey@0"
I20260808 02:18:11.505460 31955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "9058b120d2f948fabde6d2ccf5c457cb"
format_stamp: "Formatted at 2026-08-08 02:18:11 on dist-test-slave-1vmg"
server_key: "27dca86b495c999631e4fd754eeb47ca"
server_key_iv: "4b29e04e03bab181e6629fa20d29ebba"
server_key_version: "encryptionkey@0"
I20260808 02:18:11.509794 31955 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 02:18:11.512945 31978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:11.514401 31955 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:18:11.514555 31955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "9058b120d2f948fabde6d2ccf5c457cb"
format_stamp: "Formatted at 2026-08-08 02:18:11 on dist-test-slave-1vmg"
server_key: "27dca86b495c999631e4fd754eeb47ca"
server_key_iv: "4b29e04e03bab181e6629fa20d29ebba"
server_key_version: "encryptionkey@0"
I20260808 02:18:11.514695 31955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:11.520005 31955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:11.520746 31955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:11.520936 31955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:11.521570 31955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:11.522648 31955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:11.522722 31955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:11.522791 31955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:11.522819 31955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:11.532554 31955 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:35519
I20260808 02:18:11.532596 32147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:35519 every 8 connection(s)
I20260808 02:18:11.533627 31955 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:18:11.537361 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 31955
I20260808 02:18:11.537485 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:18:11.537760 24895 external_mini_cluster.cc:1428] Setting key 0df682416376b3bc1bced75f64c16de0
I20260808 02:18:11.540390 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:45977
--builtin_ntp_servers=127.24.79.212:40341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 02:18:11.541287 31728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:11.535762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:49705 (local address 127.24.79.254:45977)
0808 02:18:11.535943 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:11.535948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:11.537261 (+ 1313us) server_negotiation.cc:407] Connection header received
0808 02:18:11.538886 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:11.538890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:11.538965 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:11.539063 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:11.540690 (+ 1627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:11.540695 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:11.540698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:18:11.540718 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:11.540746 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:18:11.540757 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:11.540876 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:11.541074 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 02:18:11.541138 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 02:18:11.543422 32150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:11.536209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45977 (local address 127.24.79.194:49705)
0808 02:18:11.537019 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:11.537087 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:11.538651 (+ 1564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:11.539339 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:11.539351 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:11.539490 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:11.540296 (+ 806us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:18:11.540298 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:11.540351 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:18:11.540448 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:11.540891 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:11.540903 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:11.540910 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 02:18:11.541114 (+ 204us) client_negotiation.cc:240] Negotiation successful
0808 02:18:11.541394 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":137,"threads_started":1}
I20260808 02:18:11.545220 32148 heartbeater.cc:347] Connected to a master server at 127.24.79.254:45977
I20260808 02:18:11.545596 32148 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:11.546231 32148 heartbeater.cc:511] Master 127.24.79.254:45977 requested a full tablet report, sending...
I20260808 02:18:11.547974 31674 ts_manager.cc:194] Registered new tserver with Master: 9058b120d2f948fabde6d2ccf5c457cb (127.24.79.194:35519)
W20260808 02:18:11.666172 32153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:11.666502 32153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:11.666545 32153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:11.670677 32153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:11.670746 32153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:11.670806 32153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:18:11.675822 32153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:45977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:11.677038 32153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:11.678543 32153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:11.689121 32165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:11.689112 32169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:11.689137 32164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:11.689565 32153 server_base.cc:1034] running on GCE node
I20260808 02:18:11.689991 32153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:11.690703 32153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:11.691880 32153 hybrid_clock.cc:648] HybridClock initialized: now 1786155491691842 us; error 48 us; skew 500 ppm
I20260808 02:18:11.694013 32153 webserver.cc:460] Webserver started at http://127.24.79.195:34933/ using document root <none> and password file <none>
I20260808 02:18:11.694607 32153 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:11.694682 32153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:11.694869 32153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:11.697157 32153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "0664368f7d804e09ac41cc29383e52f0"
format_stamp: "Formatted at 2026-08-08 02:18:11 on dist-test-slave-1vmg"
server_key: "a1e7181f3f22f71d4917b351c534833e"
server_key_iv: "293f3a7fb13ccfd56307617d66e6be68"
server_key_version: "encryptionkey@0"
I20260808 02:18:11.697717 32153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "0664368f7d804e09ac41cc29383e52f0"
format_stamp: "Formatted at 2026-08-08 02:18:11 on dist-test-slave-1vmg"
server_key: "a1e7181f3f22f71d4917b351c534833e"
server_key_iv: "293f3a7fb13ccfd56307617d66e6be68"
server_key_version: "encryptionkey@0"
I20260808 02:18:11.702200 32153 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:18:11.705487 32176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:11.707024 32153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:11.707173 32153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "0664368f7d804e09ac41cc29383e52f0"
format_stamp: "Formatted at 2026-08-08 02:18:11 on dist-test-slave-1vmg"
server_key: "a1e7181f3f22f71d4917b351c534833e"
server_key_iv: "293f3a7fb13ccfd56307617d66e6be68"
server_key_version: "encryptionkey@0"
I20260808 02:18:11.707353 32153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:11.712775 32153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:11.713533 32153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:11.713727 32153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:11.714393 32153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:11.715554 32153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:11.715610 32153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:11.715654 32153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:11.715684 32153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:11.726528 32153 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:44169
I20260808 02:18:11.726593 32339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:44169 every 8 connection(s)
I20260808 02:18:11.727737 32153 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:18:11.729555 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 32153
I20260808 02:18:11.729699 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:18:11.730060 24895 external_mini_cluster.cc:1428] Setting key 8bcd32351508dd37633d997bef1ea914
I20260808 02:18:11.736272 31728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:11.730323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:39535 (local address 127.24.79.254:45977)
0808 02:18:11.730473 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:11.730477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:11.732939 (+ 2462us) server_negotiation.cc:407] Connection header received
0808 02:18:11.734417 (+ 1478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:11.734420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:11.734464 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:11.734548 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:11.735869 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:11.735872 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:11.735874 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:18:11.735889 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:11.735912 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:18:11.735920 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:11.736005 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:11.736101 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 02:18:11.736145 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:18:11.737332 32346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:11.730969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45977 (local address 127.24.79.195:39535)
0808 02:18:11.732787 (+ 1818us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:11.732847 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:11.734154 (+ 1307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:11.734784 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:11.734792 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:11.734891 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:11.735565 (+ 674us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:18:11.735566 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:11.735614 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:18:11.735699 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:11.735993 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:11.735998 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:11.736002 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:18:11.736146 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 02:18:11.736457 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":193,"threads_started":1}
I20260808 02:18:11.738692 32343 heartbeater.cc:347] Connected to a master server at 127.24.79.254:45977
I20260808 02:18:11.739014 32343 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:11.739494 32343 heartbeater.cc:511] Master 127.24.79.254:45977 requested a full tablet report, sending...
I20260808 02:18:11.740846 31674 ts_manager.cc:194] Registered new tserver with Master: 0664368f7d804e09ac41cc29383e52f0 (127.24.79.195:44169)
I20260808 02:18:11.745498 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:18:11.750303 31728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:11.747772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38978 (local address 127.24.79.254:45977)
0808 02:18:11.747924 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:11.747928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:11.748202 (+ 274us) server_negotiation.cc:407] Connection header received
0808 02:18:11.748426 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:11.748429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:11.748473 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:11.748550 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:11.749774 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:11.749778 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:11.749780 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:18:11.749794 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:11.749819 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:18:11.749828 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:11.749915 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:11.750101 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 02:18:11.750159 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 02:18:11.753988 31674 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38978:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:18:11.757963 31674 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:18:11.773334 32362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:11.770233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:35519 (local address 127.0.0.1:34098)
0808 02:18:11.770709 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:11.770721 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:11.770790 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:11.771618 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:11.771626 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:11.771711 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:11.772394 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:18:11.772396 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:11.772420 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:18:11.772506 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:11.772949 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:11.772959 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:11.772966 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 02:18:11.773077 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 02:18:11.773156 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":89,"threads_started":1}
I20260808 02:18:11.773545 32361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:11.769633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57792 (local address 127.24.79.195:44169)
0808 02:18:11.770268 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:11.770281 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:11.770512 (+ 231us) server_negotiation.cc:407] Connection header received
0808 02:18:11.770737 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:11.770746 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:11.770984 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:11.771081 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:11.772944 (+ 1863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:11.772950 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:11.772961 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:18:11.772975 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:11.773020 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:18:11.773063 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:11.773138 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:11.773318 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 02:18:11.773396 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":164,"threads_started":1}
I20260808 02:18:11.773396 32360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:11.769218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52562 (local address 127.24.79.193:46389)
0808 02:18:11.769885 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:11.769898 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:11.770401 (+ 503us) server_negotiation.cc:407] Connection header received
0808 02:18:11.770649 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:11.770656 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:11.770898 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:11.771084 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:11.772697 (+ 1613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:11.772703 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:11.772717 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:18:11.772734 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:11.772782 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:18:11.772821 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:11.772925 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:11.773139 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 02:18:11.773227 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":111,"threads_started":1}
I20260808 02:18:11.773943 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:11.769154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:44169 (local address 127.0.0.1:57792)
0808 02:18:11.770404 (+ 1250us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:11.770418 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:11.770587 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:11.771754 (+ 1167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:11.771757 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:11.771768 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:11.772770 (+ 1002us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:18:11.772772 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:11.772782 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:18:11.772805 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:11.773208 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:11.773211 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:11.773213 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 02:18:11.773781 (+ 568us) client_negotiation.cc:240] Negotiation successful
0808 02:18:11.773822 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1189,"thread_start_us":127,"threads_started":1}
I20260808 02:18:11.773979 32359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:11.769077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:46389 (local address 127.0.0.1:52562)
0808 02:18:11.770240 (+ 1163us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:11.770307 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:11.770494 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:11.771786 (+ 1292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:11.771789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:11.771805 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:11.772488 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:18:11.772490 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:11.772503 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:18:11.772537 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:11.772995 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:11.772998 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:11.773000 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 02:18:11.773842 (+ 842us) client_negotiation.cc:240] Negotiation successful
0808 02:18:11.773882 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":894,"thread_start_us":146,"threads_started":1}
I20260808 02:18:11.774035 32363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:11.770692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34098 (local address 127.24.79.194:35519)
0808 02:18:11.771179 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:11.771191 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:11.771214 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:18:11.771266 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:11.771272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:11.771436 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:11.771577 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:11.772671 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:11.772679 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:11.772693 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:18:11.772705 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:11.772757 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:18:11.772794 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:11.772907 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:11.773824 (+ 917us) server_negotiation.cc:299] Negotiation successful
0808 02:18:11.773896 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":138,"threads_started":1}
I20260808 02:18:11.777083 32248 tablet_service.cc:1467] Processing CreateTablet for tablet 77ed3b67d4ba47deb703a7156df9e882 (DEFAULT_TABLE table=test-table [id=48fb1f7cbed74fb1a7aa6bab6fe005dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:11.777072 31854 tablet_service.cc:1467] Processing CreateTablet for tablet 77ed3b67d4ba47deb703a7156df9e882 (DEFAULT_TABLE table=test-table [id=48fb1f7cbed74fb1a7aa6bab6fe005dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:11.777081 32051 tablet_service.cc:1467] Processing CreateTablet for tablet 77ed3b67d4ba47deb703a7156df9e882 (DEFAULT_TABLE table=test-table [id=48fb1f7cbed74fb1a7aa6bab6fe005dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:11.778259 31854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77ed3b67d4ba47deb703a7156df9e882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:11.778259 32051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77ed3b67d4ba47deb703a7156df9e882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:11.778281 32248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77ed3b67d4ba47deb703a7156df9e882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:11.787249 32367 tablet_bootstrap.cc:492] T 77ed3b67d4ba47deb703a7156df9e882 P 9058b120d2f948fabde6d2ccf5c457cb: Bootstrap starting.
I20260808 02:18:11.789417 32367 tablet_bootstrap.cc:654] T 77ed3b67d4ba47deb703a7156df9e882 P 9058b120d2f948fabde6d2ccf5c457cb: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:11.790086 32366 tablet_bootstrap.cc:492] T 77ed3b67d4ba47deb703a7156df9e882 P b94f77d994be40569fa84a56d9ef1324: Bootstrap starting.
I20260808 02:18:11.790138 32364 tablet_bootstrap.cc:492] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0: Bootstrap starting.
I20260808 02:18:11.790329 32367 log.cc:824] T 77ed3b67d4ba47deb703a7156df9e882 P 9058b120d2f948fabde6d2ccf5c457cb: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:11.792809 32366 tablet_bootstrap.cc:654] T 77ed3b67d4ba47deb703a7156df9e882 P b94f77d994be40569fa84a56d9ef1324: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:11.792809 32364 tablet_bootstrap.cc:654] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:11.792878 32367 tablet_bootstrap.cc:492] T 77ed3b67d4ba47deb703a7156df9e882 P 9058b120d2f948fabde6d2ccf5c457cb: No bootstrap required, opened a new log
I20260808 02:18:11.793108 32367 ts_tablet_manager.cc:1397] T 77ed3b67d4ba47deb703a7156df9e882 P 9058b120d2f948fabde6d2ccf5c457cb: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 02:18:11.793783 32364 log.cc:824] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:11.793783 32366 log.cc:824] T 77ed3b67d4ba47deb703a7156df9e882 P b94f77d994be40569fa84a56d9ef1324: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:11.796418 32366 tablet_bootstrap.cc:492] T 77ed3b67d4ba47deb703a7156df9e882 P b94f77d994be40569fa84a56d9ef1324: No bootstrap required, opened a new log
I20260808 02:18:11.796418 32364 tablet_bootstrap.cc:492] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0: No bootstrap required, opened a new log
I20260808 02:18:11.796660 32366 ts_tablet_manager.cc:1397] T 77ed3b67d4ba47deb703a7156df9e882 P b94f77d994be40569fa84a56d9ef1324: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 02:18:11.796661 32364 ts_tablet_manager.cc:1397] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 02:18:11.797775 32367 raft_consensus.cc:348] T 77ed3b67d4ba47deb703a7156df9e882 P 9058b120d2f948fabde6d2ccf5c457cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b94f77d994be40569fa84a56d9ef1324" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46389 } } peers { permanent_uuid: "0664368f7d804e09ac41cc29383e52f0" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44169 } } peers { permanent_uuid: "9058b120d2f948fabde6d2ccf5c457cb" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 35519 } }
I20260808 02:18:11.798066 32367 raft_consensus.cc:374] T 77ed3b67d4ba47deb703a7156df9e882 P 9058b120d2f948fabde6d2ccf5c457cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:11.798128 32367 raft_consensus.cc:729] T 77ed3b67d4ba47deb703a7156df9e882 P 9058b120d2f948fabde6d2ccf5c457cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9058b120d2f948fabde6d2ccf5c457cb, State: Initialized, Role: FOLLOWER
I20260808 02:18:11.798662 32367 consensus_queue.cc:260] T 77ed3b67d4ba47deb703a7156df9e882 P 9058b120d2f948fabde6d2ccf5c457cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b94f77d994be40569fa84a56d9ef1324" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46389 } } peers { permanent_uuid: "0664368f7d804e09ac41cc29383e52f0" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44169 } } peers { permanent_uuid: "9058b120d2f948fabde6d2ccf5c457cb" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 35519 } }
I20260808 02:18:11.799981 32367 ts_tablet_manager.cc:1428] T 77ed3b67d4ba47deb703a7156df9e882 P 9058b120d2f948fabde6d2ccf5c457cb: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260808 02:18:11.800414 32148 heartbeater.cc:503] Master 127.24.79.254:45977 was elected leader, sending a full tablet report...
I20260808 02:18:11.801592 32366 raft_consensus.cc:348] T 77ed3b67d4ba47deb703a7156df9e882 P b94f77d994be40569fa84a56d9ef1324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b94f77d994be40569fa84a56d9ef1324" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46389 } } peers { permanent_uuid: "0664368f7d804e09ac41cc29383e52f0" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44169 } } peers { permanent_uuid: "9058b120d2f948fabde6d2ccf5c457cb" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 35519 } }
I20260808 02:18:11.801592 32364 raft_consensus.cc:348] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b94f77d994be40569fa84a56d9ef1324" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46389 } } peers { permanent_uuid: "0664368f7d804e09ac41cc29383e52f0" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44169 } } peers { permanent_uuid: "9058b120d2f948fabde6d2ccf5c457cb" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 35519 } }
I20260808 02:18:11.801913 32364 raft_consensus.cc:374] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:11.801913 32366 raft_consensus.cc:374] T 77ed3b67d4ba47deb703a7156df9e882 P b94f77d994be40569fa84a56d9ef1324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:11.801987 32364 raft_consensus.cc:729] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0664368f7d804e09ac41cc29383e52f0, State: Initialized, Role: FOLLOWER
I20260808 02:18:11.801987 32366 raft_consensus.cc:729] T 77ed3b67d4ba47deb703a7156df9e882 P b94f77d994be40569fa84a56d9ef1324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b94f77d994be40569fa84a56d9ef1324, State: Initialized, Role: FOLLOWER
I20260808 02:18:11.802523 32366 consensus_queue.cc:260] T 77ed3b67d4ba47deb703a7156df9e882 P b94f77d994be40569fa84a56d9ef1324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b94f77d994be40569fa84a56d9ef1324" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46389 } } peers { permanent_uuid: "0664368f7d804e09ac41cc29383e52f0" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44169 } } peers { permanent_uuid: "9058b120d2f948fabde6d2ccf5c457cb" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 35519 } }
I20260808 02:18:11.802523 32364 consensus_queue.cc:260] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b94f77d994be40569fa84a56d9ef1324" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46389 } } peers { permanent_uuid: "0664368f7d804e09ac41cc29383e52f0" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44169 } } peers { permanent_uuid: "9058b120d2f948fabde6d2ccf5c457cb" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 35519 } }
I20260808 02:18:11.803611 31952 heartbeater.cc:503] Master 127.24.79.254:45977 was elected leader, sending a full tablet report...
I20260808 02:18:11.803756 32364 ts_tablet_manager.cc:1428] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260808 02:18:11.803759 32366 ts_tablet_manager.cc:1428] T 77ed3b67d4ba47deb703a7156df9e882 P b94f77d994be40569fa84a56d9ef1324: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260808 02:18:11.804241 32343 heartbeater.cc:503] Master 127.24.79.254:45977 was elected leader, sending a full tablet report...
W20260808 02:18:11.837152 31953 tablet.cc:2369] T 77ed3b67d4ba47deb703a7156df9e882 P b94f77d994be40569fa84a56d9ef1324: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:18:11.961676 32372 raft_consensus.cc:482] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:18:11.961951 32372 raft_consensus.cc:504] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b94f77d994be40569fa84a56d9ef1324" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46389 } } peers { permanent_uuid: "0664368f7d804e09ac41cc29383e52f0" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44169 } } peers { permanent_uuid: "9058b120d2f948fabde6d2ccf5c457cb" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 35519 } }
I20260808 02:18:11.963364 32372 leader_election.cc:288] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b94f77d994be40569fa84a56d9ef1324 (127.24.79.193:46389), 9058b120d2f948fabde6d2ccf5c457cb (127.24.79.194:35519)
I20260808 02:18:11.965385 32346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:11.963691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:35519 (local address 127.24.79.195:37601)
0808 02:18:11.963855 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:11.963872 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:11.963972 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:11.964269 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:11.964272 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:11.964285 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:11.964828 (+ 543us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:18:11.964830 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:11.964841 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:18:11.964867 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:11.965136 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:11.965140 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:11.965142 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 02:18:11.965210 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 02:18:11.965245 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 02:18:11.965412 32363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:11.963803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:37601 (local address 127.24.79.194:35519)
0808 02:18:11.963924 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:11.963928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:11.963943 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:18:11.964086 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:11.964089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:11.964140 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:11.964222 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:11.964997 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:11.965001 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:11.965003 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:18:11.965016 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:11.965037 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:18:11.965047 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:11.965138 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:11.965225 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 02:18:11.965273 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 02:18:11.965497 32360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:11.963833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:52089 (local address 127.24.79.193:46389)
0808 02:18:11.963975 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:11.963979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:11.964283 (+ 304us) server_negotiation.cc:407] Connection header received
0808 02:18:11.964367 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:11.964369 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:11.964403 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:11.964465 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:11.965145 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:11.965149 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:11.965151 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:18:11.965166 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:11.965188 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:18:11.965198 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:11.965284 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:11.965352 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 02:18:11.965387 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 02:18:11.965714 32374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:11.963691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:46389 (local address 127.24.79.195:52089)
0808 02:18:11.964174 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:11.964185 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:11.964275 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:11.964513 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:11.964516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:11.964532 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:11.964944 (+ 412us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:18:11.964946 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:11.964957 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:18:11.964983 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:11.965279 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:11.965281 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:11.965284 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:18:11.965354 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 02:18:11.965385 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":135,"thread_start_us":122,"threads_started":1}
I20260808 02:18:11.966233 32080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77ed3b67d4ba47deb703a7156df9e882" candidate_uuid: "0664368f7d804e09ac41cc29383e52f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9058b120d2f948fabde6d2ccf5c457cb" is_pre_election: true
I20260808 02:18:11.966296 31866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77ed3b67d4ba47deb703a7156df9e882" candidate_uuid: "0664368f7d804e09ac41cc29383e52f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b94f77d994be40569fa84a56d9ef1324" is_pre_election: true
I20260808 02:18:11.966547 32080 raft_consensus.cc:2454] T 77ed3b67d4ba47deb703a7156df9e882 P 9058b120d2f948fabde6d2ccf5c457cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0664368f7d804e09ac41cc29383e52f0 in term 0.
I20260808 02:18:11.966593 31866 raft_consensus.cc:2454] T 77ed3b67d4ba47deb703a7156df9e882 P b94f77d994be40569fa84a56d9ef1324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0664368f7d804e09ac41cc29383e52f0 in term 0.
I20260808 02:18:11.967099 32177 leader_election.cc:302] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0664368f7d804e09ac41cc29383e52f0, b94f77d994be40569fa84a56d9ef1324; no voters:
I20260808 02:18:11.967406 32372 raft_consensus.cc:2781] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:18:11.967511 32372 raft_consensus.cc:482] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:18:11.967563 32372 raft_consensus.cc:3037] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:11.968772 32372 raft_consensus.cc:504] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b94f77d994be40569fa84a56d9ef1324" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46389 } } peers { permanent_uuid: "0664368f7d804e09ac41cc29383e52f0" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44169 } } peers { permanent_uuid: "9058b120d2f948fabde6d2ccf5c457cb" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 35519 } }
I20260808 02:18:11.969197 32372 leader_election.cc:288] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0 [CANDIDATE]: Term 1 election: Requested vote from peers b94f77d994be40569fa84a56d9ef1324 (127.24.79.193:46389), 9058b120d2f948fabde6d2ccf5c457cb (127.24.79.194:35519)
I20260808 02:18:11.969578 31866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77ed3b67d4ba47deb703a7156df9e882" candidate_uuid: "0664368f7d804e09ac41cc29383e52f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b94f77d994be40569fa84a56d9ef1324"
I20260808 02:18:11.969591 32080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77ed3b67d4ba47deb703a7156df9e882" candidate_uuid: "0664368f7d804e09ac41cc29383e52f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9058b120d2f948fabde6d2ccf5c457cb"
I20260808 02:18:11.969719 31866 raft_consensus.cc:3037] T 77ed3b67d4ba47deb703a7156df9e882 P b94f77d994be40569fa84a56d9ef1324 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:11.969712 32080 raft_consensus.cc:3037] T 77ed3b67d4ba47deb703a7156df9e882 P 9058b120d2f948fabde6d2ccf5c457cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:11.970845 32080 raft_consensus.cc:2454] T 77ed3b67d4ba47deb703a7156df9e882 P 9058b120d2f948fabde6d2ccf5c457cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0664368f7d804e09ac41cc29383e52f0 in term 1.
I20260808 02:18:11.971005 31866 raft_consensus.cc:2454] T 77ed3b67d4ba47deb703a7156df9e882 P b94f77d994be40569fa84a56d9ef1324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0664368f7d804e09ac41cc29383e52f0 in term 1.
I20260808 02:18:11.971276 32181 leader_election.cc:302] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0664368f7d804e09ac41cc29383e52f0, 9058b120d2f948fabde6d2ccf5c457cb; no voters:
I20260808 02:18:11.971467 32372 raft_consensus.cc:2781] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:11.971701 32372 raft_consensus.cc:686] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0 [term 1 LEADER]: Becoming Leader. State: Replica: 0664368f7d804e09ac41cc29383e52f0, State: Running, Role: LEADER
I20260808 02:18:11.972102 32372 consensus_queue.cc:237] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b94f77d994be40569fa84a56d9ef1324" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46389 } } peers { permanent_uuid: "0664368f7d804e09ac41cc29383e52f0" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44169 } } peers { permanent_uuid: "9058b120d2f948fabde6d2ccf5c457cb" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 35519 } }
I20260808 02:18:11.975778 31672 catalog_manager.cc:5495] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0664368f7d804e09ac41cc29383e52f0 (127.24.79.195). New cstate: current_term: 1 leader_uuid: "0664368f7d804e09ac41cc29383e52f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b94f77d994be40569fa84a56d9ef1324" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0664368f7d804e09ac41cc29383e52f0" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9058b120d2f948fabde6d2ccf5c457cb" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 35519 } health_report { overall_health: UNKNOWN } } }
W20260808 02:18:11.978881 32344 tablet.cc:2369] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:18:12.015008 32361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:12.013376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57806 (local address 127.24.79.195:44169)
0808 02:18:12.013530 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:12.013535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:12.013551 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:18:12.013649 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:12.013651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:12.013719 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:12.013791 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:12.014534 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:12.014537 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:12.014539 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:18:12.014550 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:12.014570 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:18:12.014578 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:12.014659 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:12.014784 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 02:18:12.014829 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:18:12.022192 31866 raft_consensus.cc:1261] T 77ed3b67d4ba47deb703a7156df9e882 P b94f77d994be40569fa84a56d9ef1324 [term 1 FOLLOWER]: Refusing update from remote peer 0664368f7d804e09ac41cc29383e52f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:18:12.022192 32080 raft_consensus.cc:1261] T 77ed3b67d4ba47deb703a7156df9e882 P 9058b120d2f948fabde6d2ccf5c457cb [term 1 FOLLOWER]: Refusing update from remote peer 0664368f7d804e09ac41cc29383e52f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:18:12.023277 32372 consensus_queue.cc:1035] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9058b120d2f948fabde6d2ccf5c457cb" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 35519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:18:12.023547 32375 consensus_queue.cc:1035] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b94f77d994be40569fa84a56d9ef1324" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:18:12.032033 32382 mvcc.cc:205] Tried to move back new op lower bound from 7316092895313846272 to 7316092895120699392. Current Snapshot: MvccSnapshot[applied={T|T < 7316092895313846272}]
I20260808 02:18:12.032496 32386 mvcc.cc:205] Tried to move back new op lower bound from 7316092895313846272 to 7316092895120699392. Current Snapshot: MvccSnapshot[applied={T|T < 7316092895313846272}]
I20260808 02:18:12.042479 31672 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:18:12.042670 31672 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:18:12.045729 31672 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0eeefcf85ff04d86a7b253d41c28b562: Sending DeleteTablet for 3 replicas of tablet 77ed3b67d4ba47deb703a7156df9e882
I20260808 02:18:12.046854 31854 tablet_service.cc:1514] Processing DeleteTablet for tablet 77ed3b67d4ba47deb703a7156df9e882 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:12 UTC) from {username='slave'} at 127.0.0.1:52562
I20260808 02:18:12.046850 32248 tablet_service.cc:1514] Processing DeleteTablet for tablet 77ed3b67d4ba47deb703a7156df9e882 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:12 UTC) from {username='slave'} at 127.0.0.1:57792
I20260808 02:18:12.047552 32051 tablet_service.cc:1514] Processing DeleteTablet for tablet 77ed3b67d4ba47deb703a7156df9e882 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:12 UTC) from {username='slave'} at 127.0.0.1:34098
I20260808 02:18:12.047755 32397 tablet_replica.cc:330] T 77ed3b67d4ba47deb703a7156df9e882 P b94f77d994be40569fa84a56d9ef1324: stopping tablet replica
I20260808 02:18:12.048051 32399 tablet_replica.cc:330] T 77ed3b67d4ba47deb703a7156df9e882 P 9058b120d2f948fabde6d2ccf5c457cb: stopping tablet replica
I20260808 02:18:12.048175 32397 raft_consensus.cc:2229] T 77ed3b67d4ba47deb703a7156df9e882 P b94f77d994be40569fa84a56d9ef1324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:18:12.048528 32399 raft_consensus.cc:2229] T 77ed3b67d4ba47deb703a7156df9e882 P 9058b120d2f948fabde6d2ccf5c457cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:18:12.048655 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 31764
I20260808 02:18:12.048683 32397 raft_consensus.cc:2258] T 77ed3b67d4ba47deb703a7156df9e882 P b94f77d994be40569fa84a56d9ef1324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:18:12.049100 32398 tablet_replica.cc:330] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0: stopping tablet replica
I20260808 02:18:12.049490 32398 raft_consensus.cc:2229] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:18:12.050050 32399 raft_consensus.cc:2258] T 77ed3b67d4ba47deb703a7156df9e882 P 9058b120d2f948fabde6d2ccf5c457cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:18:12.051985 32398 raft_consensus.cc:2258] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:18:12.053177 32399 ts_tablet_manager.cc:1905] T 77ed3b67d4ba47deb703a7156df9e882 P 9058b120d2f948fabde6d2ccf5c457cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:18:12.053593 32398 ts_tablet_manager.cc:1905] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:18:12.056560 32399 ts_tablet_manager.cc:1918] T 77ed3b67d4ba47deb703a7156df9e882 P 9058b120d2f948fabde6d2ccf5c457cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:18:12.056799 32399 log.cc:1196] T 77ed3b67d4ba47deb703a7156df9e882 P 9058b120d2f948fabde6d2ccf5c457cb: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/77ed3b67d4ba47deb703a7156df9e882
I20260808 02:18:12.056826 32398 ts_tablet_manager.cc:1918] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:18:12.056957 32398 log.cc:1196] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/77ed3b67d4ba47deb703a7156df9e882
I20260808 02:18:12.057325 32398 ts_tablet_manager.cc:1939] T 77ed3b67d4ba47deb703a7156df9e882 P 0664368f7d804e09ac41cc29383e52f0: Deleting consensus metadata
I20260808 02:18:12.057339 32399 ts_tablet_manager.cc:1939] T 77ed3b67d4ba47deb703a7156df9e882 P 9058b120d2f948fabde6d2ccf5c457cb: Deleting consensus metadata
I20260808 02:18:12.058959 31651 catalog_manager.cc:4841] TS 9058b120d2f948fabde6d2ccf5c457cb (127.24.79.194:35519): tablet 77ed3b67d4ba47deb703a7156df9e882 (table test-table [id=48fb1f7cbed74fb1a7aa6bab6fe005dc]) successfully deleted
I20260808 02:18:12.060098 31650 catalog_manager.cc:4841] TS 0664368f7d804e09ac41cc29383e52f0 (127.24.79.195:44169): tablet 77ed3b67d4ba47deb703a7156df9e882 (table test-table [id=48fb1f7cbed74fb1a7aa6bab6fe005dc]) successfully deleted
W20260808 02:18:12.064656 31649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.79.193:46389 (error 108)
I20260808 02:18:12.065835 32362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:12.065510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:46389 (local address 127.0.0.1:52564)
0808 02:18:12.065655 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:12.065737 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.193:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 02:18:12.065914 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 31955
W20260808 02:18:12.066166 31649 catalog_manager.cc:4584] TS b94f77d994be40569fa84a56d9ef1324 (127.24.79.193:46389): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 77ed3b67d4ba47deb703a7156df9e882: Network error: Client connection negotiation failed: client connection to 127.24.79.193:46389: connect: Connection refused (error 111)
I20260808 02:18:12.082872 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 32153
I20260808 02:18:12.094837 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 31626
2026-08-08T02:18:12Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1255 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 02:18:12.124712 24895 mini_oidc.cc:152] Starting JWKS server
I20260808 02:18:12.126629 24895 webserver.cc:460] Webserver started at https://127.0.0.1:37513/ using document root <none> and password file <none>
I20260808 02:18:12.126798 24895 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 02:18:12.127125 24895 webserver.cc:460] Webserver started at http://127.0.0.1:35091/ using document root <none> and password file <none>
2026-08-08T02:18:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:18:12Z Disabled control of system clock
I20260808 02:18:12.139117 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:37023
--webserver_interface=127.24.79.254
--webserver_port=0
--builtin_ntp_servers=127.24.79.212:40189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:37023
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37513/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155464663577-24895-0/testchainca.cert with env {}
W20260808 02:18:12.261700 32406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:12.262044 32406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:12.262087 32406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:12.266752 32406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:18:12.266844 32406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:12.266858 32406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:12.266875 32406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:18:12.266891 32406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 02:18:12.266906 32406 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 02:18:12.266961 32406 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37513/jwks.json
I20260808 02:18:12.272915 32406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:37023
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:37023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37513/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155464663577-24895-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:12.274127 32406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:12.275317 32406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:12.284731 32420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:12.284734 32414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:12.285257 32406 server_base.cc:1034] running on GCE node
W20260808 02:18:12.287108 32415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:12.287971 32406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:12.289326 32406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:12.290537 32406 hybrid_clock.cc:648] HybridClock initialized: now 1786155492290491 us; error 65 us; skew 500 ppm
I20260808 02:18:12.293066 32406 webserver.cc:460] Webserver started at http://127.24.79.254:34969/ using document root <none> and password file <none>
I20260808 02:18:12.293756 32406 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:12.293834 32406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:12.294203 32406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:12.296766 32406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "d9c3e5127e2f4183bfe8e7f19164e013"
format_stamp: "Formatted at 2026-08-08 02:18:12 on dist-test-slave-1vmg"
server_key: "3e79f7fcc80ce1dbb73b92b3ba937262"
server_key_iv: "11a8cdb719e6d68e502caf08ba99c434"
server_key_version: "encryptionkey@0"
I20260808 02:18:12.297462 32406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "d9c3e5127e2f4183bfe8e7f19164e013"
format_stamp: "Formatted at 2026-08-08 02:18:12 on dist-test-slave-1vmg"
server_key: "3e79f7fcc80ce1dbb73b92b3ba937262"
server_key_iv: "11a8cdb719e6d68e502caf08ba99c434"
server_key_version: "encryptionkey@0"
I20260808 02:18:12.301843 32406 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:18:12.304802 32429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:12.306305 32406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:18:12.306452 32406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "d9c3e5127e2f4183bfe8e7f19164e013"
format_stamp: "Formatted at 2026-08-08 02:18:12 on dist-test-slave-1vmg"
server_key: "3e79f7fcc80ce1dbb73b92b3ba937262"
server_key_iv: "11a8cdb719e6d68e502caf08ba99c434"
server_key_version: "encryptionkey@0"
I20260808 02:18:12.306591 32406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:12.335216 32406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:12.336078 32406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:12.336311 32406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:12.344195 32406 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:37023
I20260808 02:18:12.344251 32504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:37023 every 8 connection(s)
I20260808 02:18:12.345311 32406 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:18:12.347343 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 32406
I20260808 02:18:12.347502 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:18:12.347888 24895 external_mini_cluster.cc:1428] Setting key 1453ddd6e226cbf19d11b89990b95848
I20260808 02:18:12.349857 32505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:12.356664 32505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9c3e5127e2f4183bfe8e7f19164e013: Bootstrap starting.
I20260808 02:18:12.358465 32509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:12.349527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35060 (local address 127.24.79.254:37023)
0808 02:18:12.350371 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:12.350392 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:12.350436 (+ 44us) server_negotiation.cc:407] Connection header received
0808 02:18:12.351223 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:12.351245 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:12.351702 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:12.352077 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:12.352554 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:12.353581 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:12.354528 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:12.355106 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:12.355271 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:12.356439 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:12.356446 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:12.356464 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:18:12.356956 (+ 492us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:12.357026 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:18:12.357051 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:12.357163 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:12.357325 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 02:18:12.357633 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":187,"threads_started":1}
I20260808 02:18:12.359526 32505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9c3e5127e2f4183bfe8e7f19164e013: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:12.360394 32505 log.cc:824] T 00000000000000000000000000000000 P d9c3e5127e2f4183bfe8e7f19164e013: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:12.361868 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:18:12.363031 32505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9c3e5127e2f4183bfe8e7f19164e013: No bootstrap required, opened a new log
I20260808 02:18:12.367528 32505 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9c3e5127e2f4183bfe8e7f19164e013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9c3e5127e2f4183bfe8e7f19164e013" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37023 } }
I20260808 02:18:12.367843 32505 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9c3e5127e2f4183bfe8e7f19164e013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:12.367910 32505 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9c3e5127e2f4183bfe8e7f19164e013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9c3e5127e2f4183bfe8e7f19164e013, State: Initialized, Role: FOLLOWER
I20260808 02:18:12.368403 32505 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9c3e5127e2f4183bfe8e7f19164e013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9c3e5127e2f4183bfe8e7f19164e013" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37023 } }
I20260808 02:18:12.368556 32505 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9c3e5127e2f4183bfe8e7f19164e013 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:18:12.368629 32505 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9c3e5127e2f4183bfe8e7f19164e013 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:18:12.368739 32505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9c3e5127e2f4183bfe8e7f19164e013 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:12.370067 32505 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9c3e5127e2f4183bfe8e7f19164e013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9c3e5127e2f4183bfe8e7f19164e013" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37023 } }
I20260808 02:18:12.370514 32505 leader_election.cc:302] T 00000000000000000000000000000000 P d9c3e5127e2f4183bfe8e7f19164e013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9c3e5127e2f4183bfe8e7f19164e013; no voters:
I20260808 02:18:12.370860 32505 leader_election.cc:288] T 00000000000000000000000000000000 P d9c3e5127e2f4183bfe8e7f19164e013 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:18:12.371011 32511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9c3e5127e2f4183bfe8e7f19164e013 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:12.371248 32511 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9c3e5127e2f4183bfe8e7f19164e013 [term 1 LEADER]: Becoming Leader. State: Replica: d9c3e5127e2f4183bfe8e7f19164e013, State: Running, Role: LEADER
I20260808 02:18:12.371610 32511 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9c3e5127e2f4183bfe8e7f19164e013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9c3e5127e2f4183bfe8e7f19164e013" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37023 } }
I20260808 02:18:12.372277 32505 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9c3e5127e2f4183bfe8e7f19164e013 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:18:12.373342 32512 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9c3e5127e2f4183bfe8e7f19164e013 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9c3e5127e2f4183bfe8e7f19164e013" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9c3e5127e2f4183bfe8e7f19164e013" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37023 } } }
I20260808 02:18:12.373349 32513 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9c3e5127e2f4183bfe8e7f19164e013 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9c3e5127e2f4183bfe8e7f19164e013. Latest consensus state: current_term: 1 leader_uuid: "d9c3e5127e2f4183bfe8e7f19164e013" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9c3e5127e2f4183bfe8e7f19164e013" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37023 } } }
I20260808 02:18:12.373502 32513 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9c3e5127e2f4183bfe8e7f19164e013 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:12.373502 32512 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9c3e5127e2f4183bfe8e7f19164e013 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:12.373996 32521 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:18:12.377029 32521 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:18:12.383637 32521 catalog_manager.cc:1347] Generated new cluster ID: 3162b2702fd74c5c96f1b7222b013759
I20260808 02:18:12.383747 32521 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:18:12.405934 32521 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:18:12.407382 32521 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:18:12.416211 32521 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9c3e5127e2f4183bfe8e7f19164e013: Generated new TSK 0
I20260808 02:18:12.417243 32521 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 02:18:12.453857 32546 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 02:18:12.454032 32546 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 02:18:12.454659 32509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:12.435899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35062 (local address 127.24.79.254:37023)
0808 02:18:12.436050 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:12.436054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:12.436334 (+ 280us) server_negotiation.cc:407] Connection header received
0808 02:18:12.436565 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:12.436569 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:12.436641 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:12.436786 (+ 145us) server_negotiation.cc:226] Negotiated authn=JWT
0808 02:18:12.437014 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:12.437849 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:12.438842 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:12.439285 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:12.439408 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:12.439861 (+ 453us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 02:18:12.454171 (+ 14310us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37513/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 02:18:12.454501 (+ 330us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35062: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37513/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 02:18:12.454777 32509 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35062: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37513/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 02:18:12.456111 32543 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:12.435748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:37023 (local address 127.0.0.1:35062)
0808 02:18:12.436237 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:12.436251 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:12.436381 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:12.436779 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:12.436782 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:12.436790 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 02:18:12.436910 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:12.436915 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:12.438038 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:12.438042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:12.438706 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:12.438717 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:12.439433 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:12.439436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:12.439521 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:12.439589 (+ 68us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 02:18:12.454470 (+ 14881us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37513/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 02:18:12.454715 (+ 245us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.79.254:37023: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37513/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":116,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.24.79.254:37023: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37513/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 02:18:12.456974 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 32406
2026-08-08T02:18:12Z chronyd exiting
I20260808 02:18:12.503526 24895 test_util.cc:175] -----------------------------------------------
I20260808 02:18:12.503644 24895 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155464663577-24895-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (380 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 02:18:12.504878 24895 mini_oidc.cc:152] Starting JWKS server
I20260808 02:18:12.505781 24895 webserver.cc:460] Webserver started at https://127.0.0.1:33755/ using document root <none> and password file <none>
I20260808 02:18:12.505875 24895 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 02:18:12.506156 24895 webserver.cc:460] Webserver started at http://127.0.0.1:34963/ using document root <none> and password file <none>
2026-08-08T02:18:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:18:12Z Disabled control of system clock
I20260808 02:18:12.517431 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:40681
--webserver_interface=127.24.79.254
--webserver_port=0
--builtin_ntp_servers=127.24.79.212:45819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:40681
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33755/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155464663577-24895-0/testchainca.cert with env {}
W20260808 02:18:12.641669 32554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:12.642004 32554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:12.642067 32554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:12.646288 32554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:18:12.646389 32554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:12.646412 32554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:12.646435 32554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:18:12.646452 32554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 02:18:12.646474 32554 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 02:18:12.646493 32554 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33755/jwks.json
I20260808 02:18:12.652552 32554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:45819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:40681
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:40681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33755/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155464663577-24895-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:12.653872 32554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:12.655045 32554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:12.664706 32564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:12.664794 32554 server_base.cc:1034] running on GCE node
W20260808 02:18:12.664695 32562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:12.664706 32561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:12.665831 32554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:12.667275 32554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:12.668478 32554 hybrid_clock.cc:648] HybridClock initialized: now 1786155492668445 us; error 57 us; skew 500 ppm
I20260808 02:18:12.670809 32554 webserver.cc:460] Webserver started at http://127.24.79.254:44509/ using document root <none> and password file <none>
I20260808 02:18:12.671522 32554 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:12.671608 32554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:12.671837 32554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:12.674255 32554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "e0246d69f1284fbc91c221ab143b7b04"
format_stamp: "Formatted at 2026-08-08 02:18:12 on dist-test-slave-1vmg"
server_key: "a9fed0384ab50be8de5ac2303d09acb8"
server_key_iv: "192fd7803d505c2ae4a5eddf9f658c34"
server_key_version: "encryptionkey@0"
I20260808 02:18:12.674947 32554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "e0246d69f1284fbc91c221ab143b7b04"
format_stamp: "Formatted at 2026-08-08 02:18:12 on dist-test-slave-1vmg"
server_key: "a9fed0384ab50be8de5ac2303d09acb8"
server_key_iv: "192fd7803d505c2ae4a5eddf9f658c34"
server_key_version: "encryptionkey@0"
I20260808 02:18:12.679414 32554 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:18:12.682478 32574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:12.684274 32554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:12.684456 32554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "e0246d69f1284fbc91c221ab143b7b04"
format_stamp: "Formatted at 2026-08-08 02:18:12 on dist-test-slave-1vmg"
server_key: "a9fed0384ab50be8de5ac2303d09acb8"
server_key_iv: "192fd7803d505c2ae4a5eddf9f658c34"
server_key_version: "encryptionkey@0"
I20260808 02:18:12.684605 32554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:12.700114 32554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:12.701012 32554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:12.701243 32554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:12.709723 32554 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:40681
I20260808 02:18:12.709769 32651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:40681 every 8 connection(s)
I20260808 02:18:12.710848 32554 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:18:12.714672 32655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:12.715626 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 32554
I20260808 02:18:12.715749 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:18:12.716080 24895 external_mini_cluster.cc:1428] Setting key 83d4fa12609f21c2f470e81a17238692
I20260808 02:18:12.721765 32655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0246d69f1284fbc91c221ab143b7b04: Bootstrap starting.
I20260808 02:18:12.724416 32655 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0246d69f1284fbc91c221ab143b7b04: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:12.725230 32655 log.cc:824] T 00000000000000000000000000000000 P e0246d69f1284fbc91c221ab143b7b04: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:12.726694 32659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:12.717721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52618 (local address 127.24.79.254:40681)
0808 02:18:12.718478 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:12.718496 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:12.718537 (+ 41us) server_negotiation.cc:407] Connection header received
0808 02:18:12.719318 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:12.719341 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:12.719864 (+ 523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:12.720216 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:12.720699 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:12.721719 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:12.722757 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:12.723315 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:12.723478 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:12.724543 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:12.724550 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:12.724571 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:18:12.725054 (+ 483us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:12.725132 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:18:12.725158 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:12.725259 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:12.725421 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 02:18:12.725739 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":162,"threads_started":1}
I20260808 02:18:12.727723 32655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0246d69f1284fbc91c221ab143b7b04: No bootstrap required, opened a new log
I20260808 02:18:12.729893 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:18:12.731896 32655 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0246d69f1284fbc91c221ab143b7b04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0246d69f1284fbc91c221ab143b7b04" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 40681 } }
I20260808 02:18:12.732203 32655 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0246d69f1284fbc91c221ab143b7b04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:12.732265 32655 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0246d69f1284fbc91c221ab143b7b04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0246d69f1284fbc91c221ab143b7b04, State: Initialized, Role: FOLLOWER
I20260808 02:18:12.732726 32655 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0246d69f1284fbc91c221ab143b7b04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0246d69f1284fbc91c221ab143b7b04" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 40681 } }
I20260808 02:18:12.732869 32655 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0246d69f1284fbc91c221ab143b7b04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:18:12.732939 32655 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0246d69f1284fbc91c221ab143b7b04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:18:12.733050 32655 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0246d69f1284fbc91c221ab143b7b04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:12.734108 32655 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0246d69f1284fbc91c221ab143b7b04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0246d69f1284fbc91c221ab143b7b04" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 40681 } }
I20260808 02:18:12.734488 32655 leader_election.cc:302] T 00000000000000000000000000000000 P e0246d69f1284fbc91c221ab143b7b04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0246d69f1284fbc91c221ab143b7b04; no voters:
I20260808 02:18:12.734826 32655 leader_election.cc:288] T 00000000000000000000000000000000 P e0246d69f1284fbc91c221ab143b7b04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:18:12.735064 32662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0246d69f1284fbc91c221ab143b7b04 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:12.735370 32662 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0246d69f1284fbc91c221ab143b7b04 [term 1 LEADER]: Becoming Leader. State: Replica: e0246d69f1284fbc91c221ab143b7b04, State: Running, Role: LEADER
I20260808 02:18:12.735726 32662 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0246d69f1284fbc91c221ab143b7b04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0246d69f1284fbc91c221ab143b7b04" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 40681 } }
I20260808 02:18:12.736207 32655 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0246d69f1284fbc91c221ab143b7b04 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:18:12.737488 32666 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0246d69f1284fbc91c221ab143b7b04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0246d69f1284fbc91c221ab143b7b04. Latest consensus state: current_term: 1 leader_uuid: "e0246d69f1284fbc91c221ab143b7b04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0246d69f1284fbc91c221ab143b7b04" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 40681 } } }
I20260808 02:18:12.737620 32666 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0246d69f1284fbc91c221ab143b7b04 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:12.737957 32665 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0246d69f1284fbc91c221ab143b7b04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0246d69f1284fbc91c221ab143b7b04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0246d69f1284fbc91c221ab143b7b04" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 40681 } } }
I20260808 02:18:12.738102 32665 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0246d69f1284fbc91c221ab143b7b04 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:12.738063 32676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:18:12.742107 32676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:18:12.748605 32676 catalog_manager.cc:1347] Generated new cluster ID: 52e4c58264644ba79e6eec4875e3326f
I20260808 02:18:12.748706 32676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:18:12.765657 32676 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:18:12.766768 32676 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:18:12.775285 32676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0246d69f1284fbc91c221ab143b7b04: Generated new TSK 0
I20260808 02:18:12.776317 32676 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 02:18:12.802825 32699 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 02:18:12.802973 32699 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 02:18:12.803802 32659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:12.787494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52634 (local address 127.24.79.254:40681)
0808 02:18:12.787658 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:12.787663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:12.787826 (+ 163us) server_negotiation.cc:407] Connection header received
0808 02:18:12.788041 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:12.788044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:12.788101 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:12.788212 (+ 111us) server_negotiation.cc:226] Negotiated authn=JWT
0808 02:18:12.788491 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:12.789175 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:12.790171 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:12.790410 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:12.790507 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:12.790871 (+ 364us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 02:18:12.803296 (+ 12425us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33755/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 02:18:12.803630 (+ 334us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52634: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33755/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 02:18:12.803892 32659 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52634: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33755/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 02:18:12.803897 32698 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:12.787345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:40681 (local address 127.0.0.1:52634)
0808 02:18:12.787718 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:12.787735 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:12.787886 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:12.788233 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:12.788237 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:12.788246 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 02:18:12.788372 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:12.788378 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:12.789304 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:12.789308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:12.790048 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:12.790059 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:12.790552 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:12.790557 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:12.790641 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:12.790671 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 02:18:12.803524 (+ 12853us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33755/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 02:18:12.803728 (+ 204us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.79.254:40681: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33755/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":112,"threads_started":1}
I20260808 02:18:12.804950 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 32554
2026-08-08T02:18:12Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (347 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 02:18:12.852221 24895 mini_oidc.cc:152] Starting JWKS server
I20260808 02:18:12.853148 24895 webserver.cc:460] Webserver started at https://127.0.0.1:42599/ using document root <none> and password file <none>
I20260808 02:18:12.853245 24895 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 02:18:12.853477 24895 webserver.cc:460] Webserver started at http://127.0.0.1:41059/ using document root <none> and password file <none>
2026-08-08T02:18:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:18:12Z Disabled control of system clock
I20260808 02:18:12.864830 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:42843
--webserver_interface=127.24.79.254
--webserver_port=0
--builtin_ntp_servers=127.24.79.212:33663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:42843
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42599/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 02:18:12.986233 32713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:12.986560 32713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:12.986625 32713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:12.990991 32713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:18:12.991106 32713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:12.991129 32713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:12.991150 32713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:18:12.991165 32713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 02:18:12.991214 32713 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 02:18:12.991233 32713 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42599/jwks.json
I20260808 02:18:12.996748 32713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:33663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:42843
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:42843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42599/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:12.998025 32713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:12.999686 32713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:13.009588 32719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:13.009572 32726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:13.009591 32722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:13.009912 32713 server_base.cc:1034] running on GCE node
I20260808 02:18:13.010761 32713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:13.012130 32713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:13.013350 32713 hybrid_clock.cc:648] HybridClock initialized: now 1786155493013331 us; error 39 us; skew 500 ppm
I20260808 02:18:13.015841 32713 webserver.cc:460] Webserver started at http://127.24.79.254:35425/ using document root <none> and password file <none>
I20260808 02:18:13.016510 32713 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:13.016592 32713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:13.016825 32713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:13.019222 32713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "97c8a5a2924546be8260bf67da2c45ba"
format_stamp: "Formatted at 2026-08-08 02:18:13 on dist-test-slave-1vmg"
server_key: "338d613bb51df5b5bda36a3b29551c25"
server_key_iv: "48039e4d79ba3555330586e923ba2915"
server_key_version: "encryptionkey@0"
I20260808 02:18:13.019870 32713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "97c8a5a2924546be8260bf67da2c45ba"
format_stamp: "Formatted at 2026-08-08 02:18:13 on dist-test-slave-1vmg"
server_key: "338d613bb51df5b5bda36a3b29551c25"
server_key_iv: "48039e4d79ba3555330586e923ba2915"
server_key_version: "encryptionkey@0"
I20260808 02:18:13.024269 32713 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:18:13.027345 32735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:13.029086 32713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:18:13.029307 32713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "97c8a5a2924546be8260bf67da2c45ba"
format_stamp: "Formatted at 2026-08-08 02:18:13 on dist-test-slave-1vmg"
server_key: "338d613bb51df5b5bda36a3b29551c25"
server_key_iv: "48039e4d79ba3555330586e923ba2915"
server_key_version: "encryptionkey@0"
I20260808 02:18:13.029455 32713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:13.046533 32713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:13.047654 32713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:13.047906 32713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:13.056546 349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:42843 every 8 connection(s)
I20260808 02:18:13.056542 32713 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:42843
I20260808 02:18:13.057755 32713 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:18:13.061587 350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:13.062659 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 32713
I20260808 02:18:13.062788 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:18:13.063165 24895 external_mini_cluster.cc:1428] Setting key 19a74b119f37df9f97894011037f360f
I20260808 02:18:13.069362 350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97c8a5a2924546be8260bf67da2c45ba: Bootstrap starting.
I20260808 02:18:13.072325 350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97c8a5a2924546be8260bf67da2c45ba: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:13.073333 350 log.cc:824] T 00000000000000000000000000000000 P 97c8a5a2924546be8260bf67da2c45ba: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:13.074553 353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:13.064579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34698 (local address 127.24.79.254:42843)
0808 02:18:13.065394 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:13.065418 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:13.065468 (+ 50us) server_negotiation.cc:407] Connection header received
0808 02:18:13.066346 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:13.066373 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:13.066941 (+ 568us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:13.067373 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:13.067858 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:13.069331 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:13.070355 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:13.071139 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:13.071314 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:13.072234 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:13.072241 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:13.072262 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:18:13.072839 (+ 577us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:13.072920 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:18:13.072949 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:13.073054 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:13.073262 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 02:18:13.073578 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":134,"threads_started":1}
I20260808 02:18:13.075908 350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97c8a5a2924546be8260bf67da2c45ba: No bootstrap required, opened a new log
I20260808 02:18:13.078157 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:18:13.080531 350 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97c8a5a2924546be8260bf67da2c45ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97c8a5a2924546be8260bf67da2c45ba" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 42843 } }
I20260808 02:18:13.080817 350 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97c8a5a2924546be8260bf67da2c45ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:13.080874 350 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97c8a5a2924546be8260bf67da2c45ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97c8a5a2924546be8260bf67da2c45ba, State: Initialized, Role: FOLLOWER
I20260808 02:18:13.081295 350 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97c8a5a2924546be8260bf67da2c45ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97c8a5a2924546be8260bf67da2c45ba" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 42843 } }
I20260808 02:18:13.081425 350 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97c8a5a2924546be8260bf67da2c45ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:18:13.081468 350 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97c8a5a2924546be8260bf67da2c45ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:18:13.081547 350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97c8a5a2924546be8260bf67da2c45ba [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:13.082718 350 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97c8a5a2924546be8260bf67da2c45ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97c8a5a2924546be8260bf67da2c45ba" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 42843 } }
I20260808 02:18:13.083156 350 leader_election.cc:302] T 00000000000000000000000000000000 P 97c8a5a2924546be8260bf67da2c45ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97c8a5a2924546be8260bf67da2c45ba; no voters:
I20260808 02:18:13.083539 350 leader_election.cc:288] T 00000000000000000000000000000000 P 97c8a5a2924546be8260bf67da2c45ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:18:13.083662 355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97c8a5a2924546be8260bf67da2c45ba [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:13.083916 355 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97c8a5a2924546be8260bf67da2c45ba [term 1 LEADER]: Becoming Leader. State: Replica: 97c8a5a2924546be8260bf67da2c45ba, State: Running, Role: LEADER
I20260808 02:18:13.084232 355 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97c8a5a2924546be8260bf67da2c45ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97c8a5a2924546be8260bf67da2c45ba" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 42843 } }
I20260808 02:18:13.084903 350 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97c8a5a2924546be8260bf67da2c45ba [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:18:13.086057 356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97c8a5a2924546be8260bf67da2c45ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97c8a5a2924546be8260bf67da2c45ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97c8a5a2924546be8260bf67da2c45ba" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 42843 } } }
I20260808 02:18:13.086084 357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97c8a5a2924546be8260bf67da2c45ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97c8a5a2924546be8260bf67da2c45ba. Latest consensus state: current_term: 1 leader_uuid: "97c8a5a2924546be8260bf67da2c45ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97c8a5a2924546be8260bf67da2c45ba" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 42843 } } }
I20260808 02:18:13.086246 356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97c8a5a2924546be8260bf67da2c45ba [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:13.086247 357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97c8a5a2924546be8260bf67da2c45ba [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:13.087566 369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:18:13.090603 369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:18:13.096670 369 catalog_manager.cc:1347] Generated new cluster ID: e2d2c6a5d5804af3886a12b3acd59a07
I20260808 02:18:13.096765 369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:18:13.110711 369 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:18:13.111871 369 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:18:13.117278 369 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97c8a5a2924546be8260bf67da2c45ba: Generated new TSK 0
I20260808 02:18:13.118094 369 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:18:13.135916 353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:13.129726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34712 (local address 127.24.79.254:42843)
0808 02:18:13.129887 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:13.129892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:13.130085 (+ 193us) server_negotiation.cc:407] Connection header received
0808 02:18:13.130292 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:13.130296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:13.130358 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:13.130470 (+ 112us) server_negotiation.cc:226] Negotiated authn=JWT
0808 02:18:13.130807 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:13.131471 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:13.132819 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:13.133203 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:13.133310 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:13.133715 (+ 405us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 02:18:13.135462 (+ 1747us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42599/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 02:18:13.135743 (+ 281us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34712: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42599/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":70}
W20260808 02:18:13.135998 353 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34712: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42599/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 02:18:13.137872 393 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:13.129543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:42843 (local address 127.0.0.1:34712)
0808 02:18:13.129939 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:13.129956 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:13.130097 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:13.130487 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:13.130491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:13.130500 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 02:18:13.130637 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:13.130644 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:13.131608 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:13.131612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:13.132641 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:13.132655 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:13.133347 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:13.133351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:13.133452 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:13.133489 (+ 37us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 02:18:13.135651 (+ 2162us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42599/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 02:18:13.137654 (+ 2003us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.79.254:42843: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42599/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":1717,"thread_start_us":166,"threads_started":1}
I20260808 02:18:13.138605 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 32713
E20260808 02:18:13.158705 394 webserver.cc:533] Webserver: sslize failed:
E20260808 02:18:13.158811 394 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T02:18:13Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (339 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T02:18:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:18:13Z Disabled control of system clock
I20260808 02:18:13.203263 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:41533
--webserver_interface=127.24.79.254
--webserver_port=0
--builtin_ntp_servers=127.24.79.212:38787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:41533
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155464663577-24895-0/testchainca.cert with env {}
W20260808 02:18:13.330153 401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:13.330502 401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:13.330572 401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:13.335438 401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:18:13.335536 401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:13.335554 401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:13.335574 401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:18:13.335592 401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 02:18:13.335611 401 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 02:18:13.335630 401 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 02:18:13.341777 401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:38787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:41533
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:41533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155464663577-24895-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:13.343143 401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:13.344303 401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:13.353793 413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:13.353793 411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:13.353797 417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:13.354209 401 server_base.cc:1034] running on GCE node
I20260808 02:18:13.354835 401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:13.356372 401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:13.357590 401 hybrid_clock.cc:648] HybridClock initialized: now 1786155493357570 us; error 35 us; skew 500 ppm
I20260808 02:18:13.359948 401 webserver.cc:460] Webserver started at http://127.24.79.254:45113/ using document root <none> and password file <none>
I20260808 02:18:13.360597 401 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:13.360667 401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:13.360882 401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:13.363243 401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "571b08425d0645f48661205f0e93a1c8"
format_stamp: "Formatted at 2026-08-08 02:18:13 on dist-test-slave-1vmg"
server_key: "845b643d32df004d8bf0d582b410a035"
server_key_iv: "66168b9e40f9843a913b86cd14650c31"
server_key_version: "encryptionkey@0"
I20260808 02:18:13.363857 401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "571b08425d0645f48661205f0e93a1c8"
format_stamp: "Formatted at 2026-08-08 02:18:13 on dist-test-slave-1vmg"
server_key: "845b643d32df004d8bf0d582b410a035"
server_key_iv: "66168b9e40f9843a913b86cd14650c31"
server_key_version: "encryptionkey@0"
I20260808 02:18:13.367939 401 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:18:13.370750 423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:13.372125 401 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:13.372268 401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "571b08425d0645f48661205f0e93a1c8"
format_stamp: "Formatted at 2026-08-08 02:18:13 on dist-test-slave-1vmg"
server_key: "845b643d32df004d8bf0d582b410a035"
server_key_iv: "66168b9e40f9843a913b86cd14650c31"
server_key_version: "encryptionkey@0"
I20260808 02:18:13.372388 401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:13.384176 401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:13.384956 401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:13.385151 401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:13.392858 401 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:41533
I20260808 02:18:13.392874 498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:41533 every 8 connection(s)
I20260808 02:18:13.394004 401 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:18:13.397563 500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:13.401072 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 401
I20260808 02:18:13.401192 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:18:13.401463 24895 external_mini_cluster.cc:1428] Setting key ae714e1718f52a67a1daffa89e3a8a1f
I20260808 02:18:13.404587 500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 571b08425d0645f48661205f0e93a1c8: Bootstrap starting.
I20260808 02:18:13.407819 500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 571b08425d0645f48661205f0e93a1c8: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:13.408779 500 log.cc:824] T 00000000000000000000000000000000 P 571b08425d0645f48661205f0e93a1c8: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:13.411569 500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 571b08425d0645f48661205f0e93a1c8: No bootstrap required, opened a new log
I20260808 02:18:13.413336 503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:13.403109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35126 (local address 127.24.79.254:41533)
0808 02:18:13.403919 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:13.403943 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:13.403996 (+ 53us) server_negotiation.cc:407] Connection header received
0808 02:18:13.404924 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:13.404955 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:13.405539 (+ 584us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:13.405985 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:13.406522 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:13.408064 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:13.409162 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:13.409790 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:13.409982 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:13.410937 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:13.410945 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:13.410966 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:18:13.411536 (+ 570us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:13.411633 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:18:13.411668 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:13.411774 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:13.411971 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 02:18:13.412294 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":141,"threads_started":1}
I20260808 02:18:13.415993 500 raft_consensus.cc:348] T 00000000000000000000000000000000 P 571b08425d0645f48661205f0e93a1c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571b08425d0645f48661205f0e93a1c8" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41533 } }
I20260808 02:18:13.416283 500 raft_consensus.cc:374] T 00000000000000000000000000000000 P 571b08425d0645f48661205f0e93a1c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:13.416345 500 raft_consensus.cc:729] T 00000000000000000000000000000000 P 571b08425d0645f48661205f0e93a1c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 571b08425d0645f48661205f0e93a1c8, State: Initialized, Role: FOLLOWER
I20260808 02:18:13.416792 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:18:13.416862 500 consensus_queue.cc:260] T 00000000000000000000000000000000 P 571b08425d0645f48661205f0e93a1c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571b08425d0645f48661205f0e93a1c8" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41533 } }
I20260808 02:18:13.417032 500 raft_consensus.cc:388] T 00000000000000000000000000000000 P 571b08425d0645f48661205f0e93a1c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:18:13.417093 500 raft_consensus.cc:482] T 00000000000000000000000000000000 P 571b08425d0645f48661205f0e93a1c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:18:13.417202 500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 571b08425d0645f48661205f0e93a1c8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:13.418525 500 raft_consensus.cc:504] T 00000000000000000000000000000000 P 571b08425d0645f48661205f0e93a1c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571b08425d0645f48661205f0e93a1c8" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41533 } }
I20260808 02:18:13.418963 500 leader_election.cc:302] T 00000000000000000000000000000000 P 571b08425d0645f48661205f0e93a1c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 571b08425d0645f48661205f0e93a1c8; no voters:
I20260808 02:18:13.419335 500 leader_election.cc:288] T 00000000000000000000000000000000 P 571b08425d0645f48661205f0e93a1c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:18:13.419471 505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 571b08425d0645f48661205f0e93a1c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:13.419741 505 raft_consensus.cc:686] T 00000000000000000000000000000000 P 571b08425d0645f48661205f0e93a1c8 [term 1 LEADER]: Becoming Leader. State: Replica: 571b08425d0645f48661205f0e93a1c8, State: Running, Role: LEADER
I20260808 02:18:13.420083 505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 571b08425d0645f48661205f0e93a1c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571b08425d0645f48661205f0e93a1c8" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41533 } }
I20260808 02:18:13.420688 500 sys_catalog.cc:564] T 00000000000000000000000000000000 P 571b08425d0645f48661205f0e93a1c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:18:13.421892 506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 571b08425d0645f48661205f0e93a1c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "571b08425d0645f48661205f0e93a1c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571b08425d0645f48661205f0e93a1c8" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41533 } } }
I20260808 02:18:13.422044 506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 571b08425d0645f48661205f0e93a1c8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:13.422348 507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 571b08425d0645f48661205f0e93a1c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 571b08425d0645f48661205f0e93a1c8. Latest consensus state: current_term: 1 leader_uuid: "571b08425d0645f48661205f0e93a1c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571b08425d0645f48661205f0e93a1c8" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41533 } } }
I20260808 02:18:13.422463 507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 571b08425d0645f48661205f0e93a1c8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:13.422552 515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:18:13.426126 515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:18:13.433198 515 catalog_manager.cc:1347] Generated new cluster ID: b65d1956ea22467b82d09c9092ac1337
I20260808 02:18:13.433301 515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:18:13.442302 515 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:18:13.443456 515 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:18:13.449924 515 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 571b08425d0645f48661205f0e93a1c8: Generated new TSK 0
I20260808 02:18:13.450796 515 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:18:13.885288 503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:13.465137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35130 (local address 127.24.79.254:41533)
0808 02:18:13.465312 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:13.465318 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:13.465524 (+ 206us) server_negotiation.cc:407] Connection header received
0808 02:18:13.465724 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:13.465726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:13.465785 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:13.465881 (+ 96us) server_negotiation.cc:226] Negotiated authn=JWT
0808 02:18:13.466155 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:13.466765 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:13.468110 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:13.468391 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:13.468494 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:13.468889 (+ 395us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 02:18:13.884850 (+415961us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 02:18:13.885149 (+ 299us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35130: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":69}
W20260808 02:18:13.885383 503 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35130: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 02:18:13.885576 542 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:13.464991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:41533 (local address 127.0.0.1:35130)
0808 02:18:13.465403 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:13.465420 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:13.465563 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:13.465894 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:13.465897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:13.465904 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0808 02:18:13.466026 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:13.466033 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:13.466886 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:13.466890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:13.467931 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:13.467946 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:13.468539 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:13.468543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:13.468662 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:13.468700 (+ 38us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 02:18:13.885145 (+416445us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 02:18:13.885363 (+ 218us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.79.254:41533: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":132,"threads_started":1}
I20260808 02:18:13.886379 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 401
2026-08-08T02:18:13Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (720 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (140 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (138 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:18:14 dist-test-slave-1vmg krb5kdc[558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:18:14 dist-test-slave-1vmg krb5kdc[558](info): set up 2 sockets
Aug 08 02:18:14 dist-test-slave-1vmg krb5kdc[558](info): commencing operation
krb5kdc: starting...
W20260808 02:18:15.323137 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:18:15 dist-test-slave-1vmg krb5kdc[558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:18:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:18:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:18:15.464479 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:37641
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:42843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:37641
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:15.589800 581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:15.590147 581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:15.590209 581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:15.594806 581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:18:15.594961 581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:15.594995 581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:15.595033 581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:18:15.595062 581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:18:15.601068 581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:42843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:37641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:37641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:15.602407 581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:15.603578 581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:18:15.613253 581 server_base.cc:1034] running on GCE node
W20260808 02:18:15.613123 593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:15.613123 591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:15.613125 588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:15.614243 581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:15.615636 581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:15.616859 581 hybrid_clock.cc:648] HybridClock initialized: now 1786155495616841 us; error 44 us; skew 500 ppm
Aug 08 02:18:15 dist-test-slave-1vmg krb5kdc[558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155495, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:15.620369 581 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:18:15.621536 581 webserver.cc:460] Webserver started at http://127.24.79.254:41697/ using document root <none> and password file <none>
I20260808 02:18:15.622298 581 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:15.622396 581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:15.622664 581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:15.625185 581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "80257599c6724eafbf00db788141d205"
format_stamp: "Formatted at 2026-08-08 02:18:15 on dist-test-slave-1vmg"
server_key: "7498c164f2aa332864cd923c5ead4785"
server_key_iv: "7212717298ecb751f8cb82ab46f11488"
server_key_version: "encryptionkey@0"
I20260808 02:18:15.625885 581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "80257599c6724eafbf00db788141d205"
format_stamp: "Formatted at 2026-08-08 02:18:15 on dist-test-slave-1vmg"
server_key: "7498c164f2aa332864cd923c5ead4785"
server_key_iv: "7212717298ecb751f8cb82ab46f11488"
server_key_version: "encryptionkey@0"
I20260808 02:18:15.631176 581 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 02:18:15.634258 606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:15.635883 581 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:15.636063 581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "80257599c6724eafbf00db788141d205"
format_stamp: "Formatted at 2026-08-08 02:18:15 on dist-test-slave-1vmg"
server_key: "7498c164f2aa332864cd923c5ead4785"
server_key_iv: "7212717298ecb751f8cb82ab46f11488"
server_key_version: "encryptionkey@0"
I20260808 02:18:15.636214 581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:15.661015 581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:15.664572 581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:15.664786 581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:15.672446 581 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:37641
I20260808 02:18:15.672510 686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:37641 every 8 connection(s)
I20260808 02:18:15.673592 581 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:18:15.673806 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 581
I20260808 02:18:15.673914 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:18:15.674207 24895 external_mini_cluster.cc:1428] Setting key 5eb2eb4ed88019024ee7b81674876daf
I20260808 02:18:15.676921 688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:15.683138 688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80257599c6724eafbf00db788141d205: Bootstrap starting.
Aug 08 02:18:15 dist-test-slave-1vmg krb5kdc[558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:18:15.685927 688 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80257599c6724eafbf00db788141d205: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:15.686858 688 log.cc:824] T 00000000000000000000000000000000 P 80257599c6724eafbf00db788141d205: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:15.689378 688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80257599c6724eafbf00db788141d205: No bootstrap required, opened a new log
I20260808 02:18:15.689740 691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:15.675741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49716 (local address 127.24.79.254:37641)
0808 02:18:15.676240 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:15.676256 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:15.676297 (+ 41us) server_negotiation.cc:407] Connection header received
0808 02:18:15.677018 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:15.677043 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:15.677435 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:15.677788 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:15.678695 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:15.679811 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:15.680700 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:15.681207 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:15.681372 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:15.684262 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:15.684269 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:15.684285 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:15.684322 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:15.686268 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:15.686864 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:15.686870 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:15.686877 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:15.687087 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:15.687448 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:15.687452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:15.687454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:15.688035 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:15.688217 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:15.688521 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 02:18:15.688882 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":159,"threads_started":1}
I20260808 02:18:15.693471 688 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80257599c6724eafbf00db788141d205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80257599c6724eafbf00db788141d205" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37641 } }
I20260808 02:18:15.693735 688 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80257599c6724eafbf00db788141d205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:15.693801 688 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80257599c6724eafbf00db788141d205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80257599c6724eafbf00db788141d205, State: Initialized, Role: FOLLOWER
I20260808 02:18:15.694283 688 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80257599c6724eafbf00db788141d205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80257599c6724eafbf00db788141d205" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37641 } }
I20260808 02:18:15.694430 688 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80257599c6724eafbf00db788141d205 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:18:15.694500 688 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80257599c6724eafbf00db788141d205 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:18:15.694609 688 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80257599c6724eafbf00db788141d205 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:15.695663 688 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80257599c6724eafbf00db788141d205 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80257599c6724eafbf00db788141d205" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37641 } }
I20260808 02:18:15.696053 688 leader_election.cc:302] T 00000000000000000000000000000000 P 80257599c6724eafbf00db788141d205 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80257599c6724eafbf00db788141d205; no voters:
I20260808 02:18:15.696455 688 leader_election.cc:288] T 00000000000000000000000000000000 P 80257599c6724eafbf00db788141d205 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:18:15.696581 694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80257599c6724eafbf00db788141d205 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:15.696769 694 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80257599c6724eafbf00db788141d205 [term 1 LEADER]: Becoming Leader. State: Replica: 80257599c6724eafbf00db788141d205, State: Running, Role: LEADER
I20260808 02:18:15.697149 694 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80257599c6724eafbf00db788141d205 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80257599c6724eafbf00db788141d205" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37641 } }
I20260808 02:18:15.697901 688 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80257599c6724eafbf00db788141d205 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:18:15.699115 696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80257599c6724eafbf00db788141d205 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80257599c6724eafbf00db788141d205" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80257599c6724eafbf00db788141d205" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37641 } } }
I20260808 02:18:15.699251 696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80257599c6724eafbf00db788141d205 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:15.700053 698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80257599c6724eafbf00db788141d205 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80257599c6724eafbf00db788141d205. Latest consensus state: current_term: 1 leader_uuid: "80257599c6724eafbf00db788141d205" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80257599c6724eafbf00db788141d205" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37641 } } }
I20260808 02:18:15.700228 698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80257599c6724eafbf00db788141d205 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:15.700430 709 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:18:15.703289 709 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:18:15.708938 709 catalog_manager.cc:1347] Generated new cluster ID: 719e9ac06dd840079d9749615516dc11
I20260808 02:18:15.709033 709 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:18:15.724536 709 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:18:15.725581 709 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:18:15.733659 709 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80257599c6724eafbf00db788141d205: Generated new TSK 0
I20260808 02:18:15.734658 709 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.79.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
I20260808 02:18:15.774673 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:37641
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:42843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:15.898094 724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:15.898452 724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:15.898518 724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:15.902856 724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:15.902961 724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:15.903050 724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:18:15.908746 724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:42843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:37641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:15.909986 724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:15.911152 724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:15.921878 735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:15.921895 740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:15.921968 724 server_base.cc:1034] running on GCE node
W20260808 02:18:15.921909 742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:15.922640 724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:15.923476 724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:15.924683 724 hybrid_clock.cc:648] HybridClock initialized: now 1786155495924661 us; error 39 us; skew 500 ppm
Aug 08 02:18:15 dist-test-slave-1vmg krb5kdc[558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155495, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:15.927901 724 init.cc:377] Logged in from keytab as kudu/127.24.79.193@KRBTEST.COM (short username kudu)
I20260808 02:18:15.928987 724 webserver.cc:460] Webserver started at http://127.24.79.193:42531/ using document root <none> and password file <none>
I20260808 02:18:15.929605 724 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:15.929679 724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:15.929915 724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:15.932299 724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "807b98ab82ad45e68dcf2d10c9121fe6"
format_stamp: "Formatted at 2026-08-08 02:18:15 on dist-test-slave-1vmg"
server_key: "be75c151774115ff4f31ae3bf543cd15"
server_key_iv: "bcfec79a45b4992ab8077a87b88afbd1"
server_key_version: "encryptionkey@0"
I20260808 02:18:15.932943 724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "807b98ab82ad45e68dcf2d10c9121fe6"
format_stamp: "Formatted at 2026-08-08 02:18:15 on dist-test-slave-1vmg"
server_key: "be75c151774115ff4f31ae3bf543cd15"
server_key_iv: "bcfec79a45b4992ab8077a87b88afbd1"
server_key_version: "encryptionkey@0"
I20260808 02:18:15.937507 724 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:18:15.940641 750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:15.942212 724 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:18:15.942359 724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "807b98ab82ad45e68dcf2d10c9121fe6"
format_stamp: "Formatted at 2026-08-08 02:18:15 on dist-test-slave-1vmg"
server_key: "be75c151774115ff4f31ae3bf543cd15"
server_key_iv: "bcfec79a45b4992ab8077a87b88afbd1"
server_key_version: "encryptionkey@0"
I20260808 02:18:15.942488 724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:15.954536 724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:15.958575 724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:15.958789 724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:15.959524 724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:15.960493 724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:15.960546 724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:15.960606 724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:15.960641 724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:15.970037 724 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:40565
I20260808 02:18:15.970160 923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:40565 every 8 connection(s)
I20260808 02:18:15.971241 724 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:18:15.973675 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 724
I20260808 02:18:15.973819 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:18:15.974144 24895 external_mini_cluster.cc:1428] Setting key 945feb7b5d6b3fd5651b8411df69e73f
Aug 08 02:18:15 dist-test-slave-1vmg krb5kdc[558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155495, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.79.194@KRBTEST.COM; defaulting to no policy
I20260808 02:18:15.987299 691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:15.973345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:37095 (local address 127.24.79.254:37641)
0808 02:18:15.973498 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:15.973502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:15.974637 (+ 1135us) server_negotiation.cc:407] Connection header received
0808 02:18:15.976303 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:15.976307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:15.976367 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:15.976445 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:15.977780 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:15.978337 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:15.979578 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:15.979852 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:15.979950 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:15.983031 (+ 3081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:15.983035 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:15.983037 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:15.983067 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:15.984709 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:15.985393 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:15.985398 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:15.985400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:15.985492 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:15.985941 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:15.985945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:15.985947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:15.986187 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:15.986283 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:15.986898 (+ 615us) server_negotiation.cc:299] Negotiation successful
0808 02:18:15.987106 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.194@KRBTEST.COM" created.
I20260808 02:18:15.988458 926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:15.973850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:37641 (local address 127.24.79.193:37095)
0808 02:18:15.974461 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:15.974514 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:15.976058 (+ 1544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:15.976643 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:15.976651 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:15.977063 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:15.977606 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:15.977621 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:15.978478 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:15.978481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:15.979440 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:15.979451 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:15.980026 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:15.980031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:15.980209 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:15.980849 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:15.980907 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:15.982777 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:15.984860 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:15.984867 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:15.984887 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:15.985273 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:15.985655 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:15.985658 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:15.985659 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:15.985838 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:15.986309 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:15.986318 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:15.986641 (+ 323us) client_negotiation.cc:769] Sending connection context
0808 02:18:15.986840 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 02:18:15.987421 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":161,"threads_started":1}
I20260808 02:18:15.990041 924 heartbeater.cc:347] Connected to a master server at 127.24.79.254:37641
I20260808 02:18:15.990330 924 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:15.991110 924 heartbeater.cc:511] Master 127.24.79.254:37641 requested a full tablet report, sending...
I20260808 02:18:15.993160 629 ts_manager.cc:194] Registered new tserver with Master: 807b98ab82ad45e68dcf2d10c9121fe6 (127.24.79.193:40565)
I20260808 02:18:15.994845 629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.193@KRBTEST.COM'} at 127.24.79.193:37095
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
I20260808 02:18:16.010619 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:37641
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:42843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:16.131430 930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:16.131738 930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:16.131783 930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:16.135993 930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:16.136078 930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:16.136143 930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:18:16.142143 930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:42843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:37641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:16.143440 930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:16.144549 930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:16.155838 940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:16.155822 936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:16.155889 930 server_base.cc:1034] running on GCE node
W20260808 02:18:16.155822 938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:16.156682 930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:16.157461 930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:16.158800 930 hybrid_clock.cc:648] HybridClock initialized: now 1786155496158676 us; error 58 us; skew 500 ppm
Aug 08 02:18:16 dist-test-slave-1vmg krb5kdc[558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155496, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:16.165946 930 init.cc:377] Logged in from keytab as kudu/127.24.79.194@KRBTEST.COM (short username kudu)
I20260808 02:18:16.167373 930 webserver.cc:460] Webserver started at http://127.24.79.194:43827/ using document root <none> and password file <none>
I20260808 02:18:16.168022 930 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:16.168104 930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:16.168340 930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:16.170677 930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "a9a300bbcbff4bf5b4253e00daa3457a"
format_stamp: "Formatted at 2026-08-08 02:18:16 on dist-test-slave-1vmg"
server_key: "87fd68079de178651141224d53f852f8"
server_key_iv: "30bde999c7c9979ff7d362e1034e7409"
server_key_version: "encryptionkey@0"
I20260808 02:18:16.171416 930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "a9a300bbcbff4bf5b4253e00daa3457a"
format_stamp: "Formatted at 2026-08-08 02:18:16 on dist-test-slave-1vmg"
server_key: "87fd68079de178651141224d53f852f8"
server_key_iv: "30bde999c7c9979ff7d362e1034e7409"
server_key_version: "encryptionkey@0"
I20260808 02:18:16.175940 930 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:18:16.179040 951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:16.180532 930 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:16.180691 930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "a9a300bbcbff4bf5b4253e00daa3457a"
format_stamp: "Formatted at 2026-08-08 02:18:16 on dist-test-slave-1vmg"
server_key: "87fd68079de178651141224d53f852f8"
server_key_iv: "30bde999c7c9979ff7d362e1034e7409"
server_key_version: "encryptionkey@0"
I20260808 02:18:16.180820 930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:16.202410 930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:16.205718 930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:16.205924 930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:16.206586 930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:16.207609 930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:16.207664 930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:16.207721 930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:16.207751 930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:16.217227 930 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:34701
I20260808 02:18:16.217279 1118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:34701 every 8 connection(s)
I20260808 02:18:16.218209 930 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:18:16.219703 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 930
I20260808 02:18:16.219817 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:18:16.220085 24895 external_mini_cluster.cc:1428] Setting key add7422db7cb524f3b6b086779d278d2
Aug 08 02:18:16 dist-test-slave-1vmg krb5kdc[558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155496, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.79.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.195@KRBTEST.COM" created.
I20260808 02:18:16.235267 691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:16.220292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:35353 (local address 127.24.79.254:37641)
0808 02:18:16.220470 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:16.220475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:16.221568 (+ 1093us) server_negotiation.cc:407] Connection header received
0808 02:18:16.223561 (+ 1993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:16.223565 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:16.223624 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:16.223711 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:16.225248 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:16.226000 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:16.227458 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:16.227811 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:16.227937 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:16.230986 (+ 3049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:16.230992 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:16.230995 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:16.231035 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:16.232575 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:16.233259 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:16.233262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:16.233264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:16.233389 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:16.233858 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:16.233861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:16.233862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:16.234104 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:16.234222 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:16.234984 (+ 762us) server_negotiation.cc:299] Negotiation successful
0808 02:18:16.235113 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 02:18:16.236395 1121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:16.220808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:37641 (local address 127.24.79.194:35353)
0808 02:18:16.221413 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:16.221478 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:16.223217 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:16.223958 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:16.223969 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:16.224474 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:16.225036 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:16.225053 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:16.226173 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:16.226176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:16.227289 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:16.227304 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:16.227972 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:16.227975 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:16.228141 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:16.228849 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:16.228868 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:16.230682 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:16.232762 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:16.232771 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:16.232793 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:16.233127 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:16.233530 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:16.233533 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:16.233536 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:16.233733 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:16.234234 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:16.234243 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:16.234613 (+ 370us) client_negotiation.cc:769] Sending connection context
0808 02:18:16.234896 (+ 283us) client_negotiation.cc:240] Negotiation successful
0808 02:18:16.235350 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":165,"threads_started":1}
I20260808 02:18:16.237859 1119 heartbeater.cc:347] Connected to a master server at 127.24.79.254:37641
I20260808 02:18:16.238144 1119 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:16.238708 1119 heartbeater.cc:511] Master 127.24.79.254:37641 requested a full tablet report, sending...
I20260808 02:18:16.240077 629 ts_manager.cc:194] Registered new tserver with Master: a9a300bbcbff4bf5b4253e00daa3457a (127.24.79.194:34701)
I20260808 02:18:16.240955 629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.194@KRBTEST.COM'} at 127.24.79.194:35353
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
I20260808 02:18:16.255008 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:37641
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:42843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:16.380398 1124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:16.380710 1124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:16.380754 1124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:16.385000 1124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:16.385161 1124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:16.385243 1124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:18:16.390949 1124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:42843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:37641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:16.392189 1124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:16.393303 1124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:16.404135 1133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:16.404345 1124 server_base.cc:1034] running on GCE node
W20260808 02:18:16.404156 1137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:16.404196 1135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:16.405025 1124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:16.405787 1124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:16.407011 1124 hybrid_clock.cc:648] HybridClock initialized: now 1786155496406985 us; error 36 us; skew 500 ppm
Aug 08 02:18:16 dist-test-slave-1vmg krb5kdc[558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155496, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:16.410193 1124 init.cc:377] Logged in from keytab as kudu/127.24.79.195@KRBTEST.COM (short username kudu)
I20260808 02:18:16.411289 1124 webserver.cc:460] Webserver started at http://127.24.79.195:42261/ using document root <none> and password file <none>
I20260808 02:18:16.412030 1124 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:16.412220 1124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:16.412536 1124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:16.414808 1124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "913451fd2bdb4ac2ae5fb82ad44c5c8d"
format_stamp: "Formatted at 2026-08-08 02:18:16 on dist-test-slave-1vmg"
server_key: "7b6f09b2d02adf53a1931c4431046599"
server_key_iv: "00617fa71d7f230b820ed30f735053b2"
server_key_version: "encryptionkey@0"
I20260808 02:18:16.415462 1124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "913451fd2bdb4ac2ae5fb82ad44c5c8d"
format_stamp: "Formatted at 2026-08-08 02:18:16 on dist-test-slave-1vmg"
server_key: "7b6f09b2d02adf53a1931c4431046599"
server_key_iv: "00617fa71d7f230b820ed30f735053b2"
server_key_version: "encryptionkey@0"
I20260808 02:18:16.419896 1124 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:18:16.422662 1145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:16.424089 1124 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:18:16.424232 1124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "913451fd2bdb4ac2ae5fb82ad44c5c8d"
format_stamp: "Formatted at 2026-08-08 02:18:16 on dist-test-slave-1vmg"
server_key: "7b6f09b2d02adf53a1931c4431046599"
server_key_iv: "00617fa71d7f230b820ed30f735053b2"
server_key_version: "encryptionkey@0"
I20260808 02:18:16.424345 1124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:16.440929 1124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:16.444433 1124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:16.444633 1124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:16.445288 1124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:16.446262 1124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:16.446312 1124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:16.446368 1124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:16.446394 1124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:16.455597 1124 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:42543
I20260808 02:18:16.455624 1304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:42543 every 8 connection(s)
I20260808 02:18:16.456584 1124 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:18:16.463871 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 1124
I20260808 02:18:16.464010 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:18:16.464339 24895 external_mini_cluster.cc:1428] Setting key 51452398fa00f5798bb9366e1b2e4fb3
Aug 08 02:18:16 dist-test-slave-1vmg krb5kdc[558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155496, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:18:16.471335 691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:16.458541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:44143 (local address 127.24.79.254:37641)
0808 02:18:16.458683 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:16.458687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:16.459674 (+ 987us) server_negotiation.cc:407] Connection header received
0808 02:18:16.461116 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:16.461120 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:16.461184 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:16.461261 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:16.462560 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:16.463180 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:16.464858 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:16.465087 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:16.465181 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:16.467972 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:16.467975 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:16.467978 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:16.468005 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:16.469309 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:16.469844 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:16.469847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:16.469848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:16.469921 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:16.470276 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:16.470278 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:16.470279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:16.470456 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:16.470530 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:16.471077 (+ 547us) server_negotiation.cc:299] Negotiation successful
0808 02:18:16.471190 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:18:16.472298 1309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:16.459004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:37641 (local address 127.24.79.195:44143)
0808 02:18:16.459513 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:16.459560 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:16.460886 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:16.461467 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:16.461474 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:16.461868 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:16.462399 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:16.462414 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:16.463317 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:16.463320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:16.464727 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:16.464739 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:16.465254 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:16.465259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:16.465439 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:16.466145 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:16.466163 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:16.467785 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:16.469436 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:16.469442 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:16.469457 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:16.469756 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:16.469996 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:16.469998 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:16.469999 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:16.470199 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:16.470531 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:16.470536 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:16.470810 (+ 274us) client_negotiation.cc:769] Sending connection context
0808 02:18:16.471036 (+ 226us) client_negotiation.cc:240] Negotiation successful
0808 02:18:16.471381 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":134,"threads_started":1}
I20260808 02:18:16.473534 1305 heartbeater.cc:347] Connected to a master server at 127.24.79.254:37641
I20260808 02:18:16.473801 1305 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:16.474409 1305 heartbeater.cc:511] Master 127.24.79.254:37641 requested a full tablet report, sending...
I20260808 02:18:16.475593 629 ts_manager.cc:194] Registered new tserver with Master: 913451fd2bdb4ac2ae5fb82ad44c5c8d (127.24.79.195:42543)
I20260808 02:18:16.476192 629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.195@KRBTEST.COM'} at 127.24.79.195:44143
I20260808 02:18:16.478775 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/test-admin.keytab.
Aug 08 02:18:16 dist-test-slave-1vmg krb5kdc[558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:16.496218 24895 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 02:18:16.499572 1317 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:16.497751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57880 (local address 127.24.79.193:40565)
0808 02:18:16.498148 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:16.498155 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:16.498184 (+ 29us) server_negotiation.cc:407] Connection header received
0808 02:18:16.498244 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:16.498250 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:16.498528 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:16.498671 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:16.499383 (+ 712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57880: BlockingRecv error: recv got EOF from 127.0.0.1:57880 (error 108)
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":89,"threads_started":1}
W20260808 02:18:16.499743 24895 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/krb5cc' of type FILE
Aug 08 02:18:16 dist-test-slave-1vmg krb5kdc[558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:16.500929 24895 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 02:18:16 dist-test-slave-1vmg krb5kdc[558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:16.501925 24895 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 02:18:16 dist-test-slave-1vmg krb5kdc[558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.193@KRBTEST.COM
I20260808 02:18:16.512413 1317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:16.503026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57886 (local address 127.24.79.193:40565)
0808 02:18:16.503229 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:16.503233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:16.503267 (+ 34us) server_negotiation.cc:407] Connection header received
0808 02:18:16.503411 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:16.503414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:16.503463 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:16.503538 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:16.504172 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:16.504975 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:16.505922 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:16.506379 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:16.506469 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:16.508981 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:16.508994 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:16.509011 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:16.509042 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:16.510630 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:16.511131 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:16.511137 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:16.511139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:16.511261 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:16.511482 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:16.511484 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:16.511485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:16.511756 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:16.511919 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:16.512090 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 02:18:16.512242 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 02:18:16.516780 1317 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:16.515680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57890 (local address 127.24.79.193:40565)
0808 02:18:16.515829 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:16.515833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:16.515900 (+ 67us) server_negotiation.cc:407] Connection header received
0808 02:18:16.516039 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:16.516042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:16.516114 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:16.516218 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:16.516680 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57890: BlockingRecv error: recv got EOF from 127.0.0.1:57890 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260808 02:18:16.517113 24895 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/krb5cc' of type FILE
Aug 08 02:18:16 dist-test-slave-1vmg krb5kdc[558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:16.518308 24895 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 02:18:16 dist-test-slave-1vmg krb5kdc[558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:16.519268 24895 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 02:18:16 dist-test-slave-1vmg krb5kdc[558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.193@KRBTEST.COM
I20260808 02:18:16.528563 1317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:16.520184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57902 (local address 127.24.79.193:40565)
0808 02:18:16.520316 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:16.520320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:16.520451 (+ 131us) server_negotiation.cc:407] Connection header received
0808 02:18:16.520558 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:16.520560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:16.520601 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:16.520673 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:16.521193 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:16.521737 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:16.522638 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:16.522883 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:16.523011 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:16.525478 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:16.525482 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:16.525485 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:16.525518 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:16.526885 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:16.527406 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:16.527409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:16.527411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:16.527497 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:16.527730 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:16.527732 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:16.527734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:16.527933 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:16.528051 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:16.528290 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 02:18:16.528427 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 02:18:16.531816 1317 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:16.530745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57906 (local address 127.24.79.193:40565)
0808 02:18:16.530882 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:16.530886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:16.530995 (+ 109us) server_negotiation.cc:407] Connection header received
0808 02:18:16.531125 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:16.531127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:16.531168 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:16.531227 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:16.531698 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57906: BlockingRecv error: recv got EOF from 127.0.0.1:57906 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 02:18:16.531973 24895 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155464663577-24895-0/krb5kdc/krb5cc' of type FILE
Aug 08 02:18:16 dist-test-slave-1vmg krb5kdc[558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:16.533031 24895 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 02:18:16 dist-test-slave-1vmg krb5kdc[558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:16.533968 24895 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 02:18:16 dist-test-slave-1vmg krb5kdc[558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.193@KRBTEST.COM
I20260808 02:18:16.543329 1317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:16.534995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57910 (local address 127.24.79.193:40565)
0808 02:18:16.535185 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:16.535188 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:16.535231 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:18:16.535355 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:16.535357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:16.535403 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:16.535472 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:16.535983 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:16.536562 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:16.537372 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:16.537614 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:16.537699 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:16.540011 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:16.540015 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:16.540017 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:16.540079 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:16.541604 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:16.542146 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:16.542149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:16.542150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:16.542237 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:16.542534 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:16.542536 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:16.542537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:16.542716 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:16.542857 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:16.543066 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 02:18:16.543177 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:18:16.544754 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 724
I20260808 02:18:16.559120 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 930
I20260808 02:18:16.582320 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 1124
I20260808 02:18:16.608234 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 581
2026-08-08T02:18:16Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2451 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:18:16 dist-test-slave-1vmg krb5kdc[1339](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:18:16 dist-test-slave-1vmg krb5kdc[1339](info): set up 2 sockets
Aug 08 02:18:16 dist-test-slave-1vmg krb5kdc[1339](info): commencing operation
krb5kdc: starting...
W20260808 02:18:17.768687 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:18:17 dist-test-slave-1vmg krb5kdc[1339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:18:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:18:17Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.254.keytab.
Entry for principal oryx/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.254.keytab.
I20260808 02:18:17.917107 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:45167
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.254.keytab
--principal=oryx/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:45167
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:18.043565 1361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:18.043900 1361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:18.043962 1361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:18.048290 1361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:18:18.048385 1361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:18.048415 1361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:18.048437 1361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:18:18.048460 1361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:18:18.054721 1361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:45167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.254.keytab
--principal=oryx/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:45167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:18.056077 1361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:18.057272 1361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:18.067169 1368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:18.067190 1374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:18.067209 1367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:18.067530 1361 server_base.cc:1034] running on GCE node
I20260808 02:18:18.068229 1361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:18.069547 1361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:18.070778 1361 hybrid_clock.cc:648] HybridClock initialized: now 1786155498070754 us; error 43 us; skew 500 ppm
Aug 08 02:18:18 dist-test-slave-1vmg krb5kdc[1339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155498, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:18.074132 1361 init.cc:377] Logged in from keytab as oryx/127.24.79.254@KRBTEST.COM (short username oryx)
I20260808 02:18:18.075356 1361 webserver.cc:460] Webserver started at http://127.24.79.254:45925/ using document root <none> and password file <none>
I20260808 02:18:18.076035 1361 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:18.076116 1361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:18.076346 1361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:18.078734 1361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "aa78ed5a894742328b0e31dda1f3499a"
format_stamp: "Formatted at 2026-08-08 02:18:18 on dist-test-slave-1vmg"
server_key: "7ac8504a2d54b57ce9a464ad80429af6"
server_key_iv: "f67d8aa9de96ab76ebc29eefc47380a2"
server_key_version: "encryptionkey@0"
I20260808 02:18:18.079478 1361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "aa78ed5a894742328b0e31dda1f3499a"
format_stamp: "Formatted at 2026-08-08 02:18:18 on dist-test-slave-1vmg"
server_key: "7ac8504a2d54b57ce9a464ad80429af6"
server_key_iv: "f67d8aa9de96ab76ebc29eefc47380a2"
server_key_version: "encryptionkey@0"
I20260808 02:18:18.084008 1361 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:18:18.087059 1385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:18.088656 1361 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:18:18.088848 1361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "aa78ed5a894742328b0e31dda1f3499a"
format_stamp: "Formatted at 2026-08-08 02:18:18 on dist-test-slave-1vmg"
server_key: "7ac8504a2d54b57ce9a464ad80429af6"
server_key_iv: "f67d8aa9de96ab76ebc29eefc47380a2"
server_key_version: "encryptionkey@0"
I20260808 02:18:18.088984 1361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:18.108461 1361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:18.112171 1361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:18.112397 1361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:18.120087 1361 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:45167
I20260808 02:18:18.120438 1464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:45167 every 8 connection(s)
I20260808 02:18:18.121265 1361 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:18:18.124776 1465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:18.126557 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 1361
I20260808 02:18:18.126682 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:18:18.126997 24895 external_mini_cluster.cc:1428] Setting key 50e27a60077e9f56c38e4e87aa68b0dc
I20260808 02:18:18.132179 1465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a: Bootstrap starting.
I20260808 02:18:18.134882 1465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:18.135993 1465 log.cc:824] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:18:18 dist-test-slave-1vmg krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM
I20260808 02:18:18.138551 1465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a: No bootstrap required, opened a new log
I20260808 02:18:18.143078 1468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:18.128730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41066 (local address 127.24.79.254:45167)
0808 02:18:18.129358 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:18.129377 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:18.129421 (+ 44us) server_negotiation.cc:407] Connection header received
0808 02:18:18.130195 (+ 774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:18.130219 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:18.130614 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:18.131039 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:18.131968 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:18.133176 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:18.134143 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:18.134678 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:18.134846 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:18.137641 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:18.137647 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:18.137666 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:18.137700 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:18.139643 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:18.140258 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:18.140264 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:18.140270 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:18.140411 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:18.140789 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:18.140793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:18.140795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:18.141287 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:18.141463 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:18.141792 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 02:18:18.142126 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":187,"threads_started":1}
I20260808 02:18:18.142974 1465 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } }
I20260808 02:18:18.143266 1465 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:18.143364 1465 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa78ed5a894742328b0e31dda1f3499a, State: Initialized, Role: FOLLOWER
I20260808 02:18:18.143893 1465 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } }
I20260808 02:18:18.144065 1465 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:18:18.144132 1465 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:18:18.144245 1465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:18.145323 1465 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } }
I20260808 02:18:18.145730 1465 leader_election.cc:302] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa78ed5a894742328b0e31dda1f3499a; no voters:
I20260808 02:18:18.146086 1465 leader_election.cc:288] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:18:18.146209 1470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:18.146492 1470 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [term 1 LEADER]: Becoming Leader. State: Replica: aa78ed5a894742328b0e31dda1f3499a, State: Running, Role: LEADER
I20260808 02:18:18.146850 1470 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } }
I20260808 02:18:18.147400 1465 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:18:18.148433 1472 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa78ed5a894742328b0e31dda1f3499a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } }
I20260808 02:18:18.148562 1472 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:18.149032 1473 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa78ed5a894742328b0e31dda1f3499a. Latest consensus state: current_term: 1 leader_uuid: "aa78ed5a894742328b0e31dda1f3499a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } }
I20260808 02:18:18.149129 1473 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [sys.catalog]: This master's current role is: LEADER
W20260808 02:18:18.152405 1495 catalog_manager.cc:1558] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:18:18.152487 1495 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:18:18.152575 1482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:18:18.153308 1482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:18:18.159303 1482 catalog_manager.cc:1347] Generated new cluster ID: 0a4525e343e5431e9aa0df4f93800b9c
I20260808 02:18:18.159407 1482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:18:18.170729 1482 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:18:18.171850 1482 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:18:18.188299 1482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a: Generated new TSK 0
I20260808 02:18:18.189206 1482 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.79.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.79.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.79.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.193.keytab.
Entry for principal oryx/127.24.79.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.193.keytab.
I20260808 02:18:18.238751 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:45167
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.193.keytab
--principal=oryx/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:18.363519 1503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:18.363862 1503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:18.363909 1503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:18.368333 1503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:18.368420 1503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:18.368484 1503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:18:18.374265 1503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.193.keytab
--principal=oryx/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:45167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:18.375615 1503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:18.376827 1503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:18.387799 1513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:18.387814 1509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:18.387814 1511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:18.388120 1503 server_base.cc:1034] running on GCE node
I20260808 02:18:18.388651 1503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:18.389413 1503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:18.390630 1503 hybrid_clock.cc:648] HybridClock initialized: now 1786155498390609 us; error 35 us; skew 500 ppm
Aug 08 02:18:18 dist-test-slave-1vmg krb5kdc[1339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155498, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:18.393977 1503 init.cc:377] Logged in from keytab as oryx/127.24.79.193@KRBTEST.COM (short username oryx)
I20260808 02:18:18.395167 1503 webserver.cc:460] Webserver started at http://127.24.79.193:33151/ using document root <none> and password file <none>
I20260808 02:18:18.395794 1503 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:18.395869 1503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:18.396103 1503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:18.398402 1503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "cbf75c267d9a4217bbf9c951246d6a93"
format_stamp: "Formatted at 2026-08-08 02:18:18 on dist-test-slave-1vmg"
server_key: "d254188a19d0de4dda986bf7c77cafb9"
server_key_iv: "8d4fbc2dcabfb854a8da6d4489016a3f"
server_key_version: "encryptionkey@0"
I20260808 02:18:18.399097 1503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "cbf75c267d9a4217bbf9c951246d6a93"
format_stamp: "Formatted at 2026-08-08 02:18:18 on dist-test-slave-1vmg"
server_key: "d254188a19d0de4dda986bf7c77cafb9"
server_key_iv: "8d4fbc2dcabfb854a8da6d4489016a3f"
server_key_version: "encryptionkey@0"
I20260808 02:18:18.403584 1503 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:18:18.406495 1526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:18.408093 1503 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:18.408262 1503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "cbf75c267d9a4217bbf9c951246d6a93"
format_stamp: "Formatted at 2026-08-08 02:18:18 on dist-test-slave-1vmg"
server_key: "d254188a19d0de4dda986bf7c77cafb9"
server_key_iv: "8d4fbc2dcabfb854a8da6d4489016a3f"
server_key_version: "encryptionkey@0"
I20260808 02:18:18.408391 1503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:18.439462 1503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:18.443565 1503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:18.443813 1503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:18.444530 1503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:18.445492 1503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:18.445546 1503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:18.445613 1503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:18.445639 1503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:18.455432 1503 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:46067
I20260808 02:18:18.455466 1687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:46067 every 8 connection(s)
I20260808 02:18:18.456470 1503 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:18:18.457919 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 1503
I20260808 02:18:18.458060 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:18:18.458495 24895 external_mini_cluster.cc:1428] Setting key f87e32a033faf467f0b241dded568593
Aug 08 02:18:18 dist-test-slave-1vmg krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155498, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.24.79.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.79.194@KRBTEST.COM" created.
I20260808 02:18:18.474215 1468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:18.458792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:40213 (local address 127.24.79.254:45167)
0808 02:18:18.459004 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:18.459009 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:18.460136 (+ 1127us) server_negotiation.cc:407] Connection header received
0808 02:18:18.461703 (+ 1567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:18.461707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:18.461772 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:18.461866 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:18.463865 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:18.464671 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:18.466021 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:18.466397 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:18.466519 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:18.469705 (+ 3186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:18.469710 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:18.469713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:18.469748 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:18.471430 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:18.472246 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:18.472250 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:18.472251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:18.472363 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:18.472835 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:18.472837 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:18.472839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:18.473066 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:18.473170 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:18.473888 (+ 718us) server_negotiation.cc:299] Negotiation successful
0808 02:18:18.474043 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 02:18:18.475152 1692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:18.459366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.193:40213)
0808 02:18:18.459967 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:18.460030 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:18.461432 (+ 1402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:18.462200 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:18.462212 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:18.462778 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:18.463642 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:18.463668 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:18.464868 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:18.464871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:18.465875 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:18.465888 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:18.466585 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:18.466589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:18.466784 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:18.467581 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:18.467601 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:18.469453 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:18.471603 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:18.471611 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:18.471657 (+ 46us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:18.471996 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:18.472488 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:18.472492 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:18.472494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:18.472730 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:18.473237 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:18.473246 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:18.473578 (+ 332us) client_negotiation.cc:769] Sending connection context
0808 02:18:18.473770 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 02:18:18.474103 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":130,"threads_started":1}
I20260808 02:18:18.476642 1689 heartbeater.cc:347] Connected to a master server at 127.24.79.254:45167
I20260808 02:18:18.476982 1689 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:18.477690 1689 heartbeater.cc:511] Master 127.24.79.254:45167 requested a full tablet report, sending...
I20260808 02:18:18.479744 1411 ts_manager.cc:194] Registered new tserver with Master: cbf75c267d9a4217bbf9c951246d6a93 (127.24.79.193:46067)
I20260808 02:18:18.481065 1411 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.79.193@KRBTEST.COM'} at 127.24.79.193:40213
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.79.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.194.keytab.
Entry for principal oryx/127.24.79.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.194.keytab.
I20260808 02:18:18.494100 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:45167
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.194.keytab
--principal=oryx/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:18.619213 1696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:18.619513 1696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:18.619557 1696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:18.623864 1696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:18.623945 1696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:18.624008 1696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:18:18.629652 1696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.194.keytab
--principal=oryx/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:45167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:18.630872 1696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:18.632076 1696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:18.642575 1705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:18.642607 1704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:18.642757 1696 server_base.cc:1034] running on GCE node
W20260808 02:18:18.642589 1708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:18.643401 1696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:18.644137 1696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:18.645340 1696 hybrid_clock.cc:648] HybridClock initialized: now 1786155498645305 us; error 50 us; skew 500 ppm
Aug 08 02:18:18 dist-test-slave-1vmg krb5kdc[1339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155498, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:18.648646 1696 init.cc:377] Logged in from keytab as oryx/127.24.79.194@KRBTEST.COM (short username oryx)
I20260808 02:18:18.649915 1696 webserver.cc:460] Webserver started at http://127.24.79.194:46813/ using document root <none> and password file <none>
I20260808 02:18:18.650576 1696 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:18.650655 1696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:18.650873 1696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:18.653141 1696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "27a7405c7485416e8f2b14a5526d9178"
format_stamp: "Formatted at 2026-08-08 02:18:18 on dist-test-slave-1vmg"
server_key: "cf8090a1c1ac858fc3575a0b01d8ff2e"
server_key_iv: "7e9c0a7075546dffa172f020b7510a9e"
server_key_version: "encryptionkey@0"
I20260808 02:18:18.653770 1696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "27a7405c7485416e8f2b14a5526d9178"
format_stamp: "Formatted at 2026-08-08 02:18:18 on dist-test-slave-1vmg"
server_key: "cf8090a1c1ac858fc3575a0b01d8ff2e"
server_key_iv: "7e9c0a7075546dffa172f020b7510a9e"
server_key_version: "encryptionkey@0"
I20260808 02:18:18.658041 1696 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 02:18:18.661195 1720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:18.662802 1696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:18:18.662992 1696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "27a7405c7485416e8f2b14a5526d9178"
format_stamp: "Formatted at 2026-08-08 02:18:18 on dist-test-slave-1vmg"
server_key: "cf8090a1c1ac858fc3575a0b01d8ff2e"
server_key_iv: "7e9c0a7075546dffa172f020b7510a9e"
server_key_version: "encryptionkey@0"
I20260808 02:18:18.663129 1696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:18.678300 1696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:18.682228 1696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:18.682440 1696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:18.683209 1696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:18.684355 1696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:18.684417 1696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:18.684465 1696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:18.684490 1696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:18.694196 1696 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:41143
I20260808 02:18:18.694226 1894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:41143 every 8 connection(s)
I20260808 02:18:18.695346 1696 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:18:18.703193 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 1696
I20260808 02:18:18.703323 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:18:18.703612 24895 external_mini_cluster.cc:1428] Setting key e5aaba8beb86afa5e97d70212bf2d504
Aug 08 02:18:18 dist-test-slave-1vmg krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155498, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM
I20260808 02:18:18.712575 1468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:18.697549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:59975 (local address 127.24.79.254:45167)
0808 02:18:18.697769 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:18.697775 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:18.698815 (+ 1040us) server_negotiation.cc:407] Connection header received
0808 02:18:18.700477 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:18.700480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:18.700539 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:18.700645 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:18.702559 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:18.703180 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:18.704760 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:18.705005 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:18.705137 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:18.708281 (+ 3144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:18.708286 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:18.708289 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:18.708327 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:18.710051 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:18.710716 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:18.710720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:18.710721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:18.710818 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:18.711275 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:18.711279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:18.711280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:18.711499 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:18.711605 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:18.712236 (+ 631us) server_negotiation.cc:299] Negotiation successful
0808 02:18:18.712386 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260808 02:18:18.713537 1897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:18.698032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.194:59975)
0808 02:18:18.698624 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:18.698679 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:18.700166 (+ 1487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:18.700897 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:18.700906 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:18.701481 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:18.702338 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:18.702362 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:18.703338 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:18.703342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:18.704626 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:18.704639 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:18.705249 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:18.705256 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:18.705474 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:18.706158 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:18.706178 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:18.708028 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:18.710229 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:18.710238 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:18.710259 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:18.710591 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:18.710954 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:18.710958 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:18.710960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:18.711163 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:18.711622 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:18.711632 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:18.711944 (+ 312us) client_negotiation.cc:769] Sending connection context
0808 02:18:18.712144 (+ 200us) client_negotiation.cc:240] Negotiation successful
0808 02:18:18.712580 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":146,"threads_started":1}
WARNING: no policy specified for oryx/127.24.79.195@KRBTEST.COM; defaulting to no policy
I20260808 02:18:18.715040 1895 heartbeater.cc:347] Connected to a master server at 127.24.79.254:45167
I20260808 02:18:18.715343 1895 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:18.715953 1895 heartbeater.cc:511] Master 127.24.79.254:45167 requested a full tablet report, sending...
I20260808 02:18:18.717240 1411 ts_manager.cc:194] Registered new tserver with Master: 27a7405c7485416e8f2b14a5526d9178 (127.24.79.194:41143)
I20260808 02:18:18.717953 1411 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.79.194@KRBTEST.COM'} at 127.24.79.194:59975
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.79.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.79.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.195.keytab.
Entry for principal oryx/127.24.79.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.195.keytab.
I20260808 02:18:18.739291 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:45167
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.195.keytab
--principal=oryx/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:18.866982 1900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:18.867359 1900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:18.867408 1900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:18.871644 1900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:18.871737 1900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:18.871799 1900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:18:18.877619 1900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.195.keytab
--principal=oryx/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:45167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:18.878836 1900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:18.879981 1900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:18.891402 1908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:18.891490 1900 server_base.cc:1034] running on GCE node
W20260808 02:18:18.891433 1906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:18.891402 1911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:18.892163 1900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:18.892974 1900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:18.894187 1900 hybrid_clock.cc:648] HybridClock initialized: now 1786155498894159 us; error 49 us; skew 500 ppm
Aug 08 02:18:18 dist-test-slave-1vmg krb5kdc[1339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155498, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:18.897287 1900 init.cc:377] Logged in from keytab as oryx/127.24.79.195@KRBTEST.COM (short username oryx)
I20260808 02:18:18.898332 1900 webserver.cc:460] Webserver started at http://127.24.79.195:34149/ using document root <none> and password file <none>
I20260808 02:18:18.898947 1900 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:18.899029 1900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:18.899240 1900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:18.901510 1900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "053e2be59b2d4d06ae90fc3bff12aebb"
format_stamp: "Formatted at 2026-08-08 02:18:18 on dist-test-slave-1vmg"
server_key: "2a1e6388e182530431a07643b14e0028"
server_key_iv: "e4bf038160bb31e741015e157bb694b7"
server_key_version: "encryptionkey@0"
I20260808 02:18:18.902098 1900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "053e2be59b2d4d06ae90fc3bff12aebb"
format_stamp: "Formatted at 2026-08-08 02:18:18 on dist-test-slave-1vmg"
server_key: "2a1e6388e182530431a07643b14e0028"
server_key_iv: "e4bf038160bb31e741015e157bb694b7"
server_key_version: "encryptionkey@0"
I20260808 02:18:18.906438 1900 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260808 02:18:18.909447 1921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:18.911095 1900 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:18.911258 1900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "053e2be59b2d4d06ae90fc3bff12aebb"
format_stamp: "Formatted at 2026-08-08 02:18:18 on dist-test-slave-1vmg"
server_key: "2a1e6388e182530431a07643b14e0028"
server_key_iv: "e4bf038160bb31e741015e157bb694b7"
server_key_version: "encryptionkey@0"
I20260808 02:18:18.911401 1900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:18.949988 1900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:18.953574 1900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:18.953768 1900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:18.954442 1900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:18.955446 1900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:18.955498 1900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:18.955554 1900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:18.955585 1900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:18.965090 1900 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:35579
I20260808 02:18:18.965118 2083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:35579 every 8 connection(s)
I20260808 02:18:18.966107 1900 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:18:18.969154 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 1900
I20260808 02:18:18.969293 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:18:18.969640 24895 external_mini_cluster.cc:1428] Setting key 003449a2cba8792e1b8a5c699b642a02
Aug 08 02:18:18 dist-test-slave-1vmg krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155498, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM
I20260808 02:18:18.982014 1468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:18.968219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:45065 (local address 127.24.79.254:45167)
0808 02:18:18.968368 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:18.968372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:18.969349 (+ 977us) server_negotiation.cc:407] Connection header received
0808 02:18:18.971074 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:18.971078 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:18.971138 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:18.971260 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:18.972815 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:18.973359 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:18.974595 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:18.974828 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:18.974976 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:18.977988 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:18.977992 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:18.977994 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:18.978026 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:18.979509 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:18.980115 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:18.980119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:18.980120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:18.980254 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:18.980654 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:18.980657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:18.980658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:18.980913 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:18.980993 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:18.981738 (+ 745us) server_negotiation.cc:299] Negotiation successful
0808 02:18:18.981879 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:18:18.983136 2086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:18.968625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.195:45065)
0808 02:18:18.969174 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:18.969224 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:18.970822 (+ 1598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:18.971449 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:18.971459 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:18.972026 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:18.972657 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:18.972674 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:18.973489 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:18.973492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:18.974450 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:18.974462 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:18.975044 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:18.975048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:18.975231 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:18.975888 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:18.975907 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:18.977761 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:18.979647 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:18.979654 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:18.979671 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:18.980006 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:18.980358 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:18.980361 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:18.980362 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:18.980559 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:18.981020 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:18.981026 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:18.981438 (+ 412us) client_negotiation.cc:769] Sending connection context
0808 02:18:18.981655 (+ 217us) client_negotiation.cc:240] Negotiation successful
0808 02:18:18.981987 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":141,"threads_started":1}
I20260808 02:18:18.984390 2084 heartbeater.cc:347] Connected to a master server at 127.24.79.254:45167
I20260808 02:18:18.984690 2084 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:18.985219 2084 heartbeater.cc:511] Master 127.24.79.254:45167 requested a full tablet report, sending...
I20260808 02:18:18.986415 1405 ts_manager.cc:194] Registered new tserver with Master: 053e2be59b2d4d06ae90fc3bff12aebb (127.24.79.195:35579)
I20260808 02:18:18.987083 1405 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.79.195@KRBTEST.COM'} at 127.24.79.195:45065
I20260808 02:18:18.996899 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.79.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.79.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.79.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.253.keytab.
Entry for principal oryx/127.24.79.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.253.keytab.
I20260808 02:18:19.031288 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.253:38013
--webserver_interface=127.24.79.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.253.keytab
--principal=oryx/127.24.79.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:45167,127.24.79.253:38013
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:19.156708 2093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:19.157061 2093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:19.157124 2093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:19.161466 2093 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:18:19.161556 2093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:19.161586 2093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:19.161609 2093 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:18:19.161646 2093 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:18:19.167728 2093 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:45167,127.24.79.253:38013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.253.keytab
--principal=oryx/127.24.79.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.253:38013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.79.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
W20260808 02:18:19.168037 2093 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.24.79.254:45167,127.24.79.253:38013'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 02:18:19.169085 2093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:19.170259 2093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:19.180090 2107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:19.180190 2102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:19.180189 2104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:19.180361 2093 server_base.cc:1034] running on GCE node
I20260808 02:18:19.181134 2093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:19.182447 2093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:19.183638 2093 hybrid_clock.cc:648] HybridClock initialized: now 1786155499183621 us; error 33 us; skew 500 ppm
Aug 08 02:18:19 dist-test-slave-1vmg krb5kdc[1339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155499, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:19.187175 2093 init.cc:377] Logged in from keytab as oryx/127.24.79.253@KRBTEST.COM (short username oryx)
I20260808 02:18:19.188323 2093 webserver.cc:460] Webserver started at http://127.24.79.253:38827/ using document root <none> and password file <none>
I20260808 02:18:19.188966 2093 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:19.189041 2093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:19.189292 2093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:19.191795 2093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/data/instance:
uuid: "5b0b6a0e3064489a90c4859f80edc3f2"
format_stamp: "Formatted at 2026-08-08 02:18:19 on dist-test-slave-1vmg"
server_key: "2d23d132a5c649b654225028dcec27e2"
server_key_iv: "8370b5e3f6f568bfacaa88acd7256e5e"
server_key_version: "encryptionkey@0"
I20260808 02:18:19.192476 2093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/wal/instance:
uuid: "5b0b6a0e3064489a90c4859f80edc3f2"
format_stamp: "Formatted at 2026-08-08 02:18:19 on dist-test-slave-1vmg"
server_key: "2d23d132a5c649b654225028dcec27e2"
server_key_iv: "8370b5e3f6f568bfacaa88acd7256e5e"
server_key_version: "encryptionkey@0"
I20260808 02:18:19.197068 2093 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:18:19.200109 2116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:19.201699 2093 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:19.201870 2093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/wal
uuid: "5b0b6a0e3064489a90c4859f80edc3f2"
format_stamp: "Formatted at 2026-08-08 02:18:19 on dist-test-slave-1vmg"
server_key: "2d23d132a5c649b654225028dcec27e2"
server_key_iv: "8370b5e3f6f568bfacaa88acd7256e5e"
server_key_version: "encryptionkey@0"
I20260808 02:18:19.202018 2093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:19.228607 2093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:19.232709 2093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:19.232949 2093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:19.241392 2093 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.253:38013
I20260808 02:18:19.241679 2192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.253:38013 every 8 connection(s)
I20260808 02:18:19.242619 2093 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/data/info.pb
I20260808 02:18:19.245779 2194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:19.250068 2194 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:18:19.250500 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 2093
Aug 08 02:18:19 dist-test-slave-1vmg krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155499, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.253@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM
Aug 08 02:18:19 dist-test-slave-1vmg krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.79.253@KRBTEST.COM
I20260808 02:18:19.264457 1468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.252389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41080 (local address 127.24.79.254:45167)
0808 02:18:19.252538 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:19.252543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:19.253029 (+ 486us) server_negotiation.cc:407] Connection header received
0808 02:18:19.253905 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:19.253908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:19.253965 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:19.254040 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:19.255716 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.256346 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.257616 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.257931 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.258052 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.260597 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:19.260601 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:19.260603 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:19.260631 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:19.262160 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.262693 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.262697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.262698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.262790 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.263184 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.263188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.263190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.263444 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:19.263541 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:19.264129 (+ 588us) server_negotiation.cc:299] Negotiation successful
0808 02:18:19.264279 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:18:19.265468 2203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.252327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.0.0.1:41080)
0808 02:18:19.252883 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:19.252924 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:19.253685 (+ 761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:19.255000 (+ 1315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:19.255006 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:19.255378 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:19.255548 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.255559 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.256508 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.256511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.257439 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.257450 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.258097 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.258100 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.258205 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.258722 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:19.258743 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:19.260430 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:19.262277 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.262288 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.262302 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.262587 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.262880 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.262883 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.262885 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.263091 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.263556 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:19.263565 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:19.263871 (+ 306us) client_negotiation.cc:769] Sending connection context
0808 02:18:19.264048 (+ 177us) client_negotiation.cc:240] Negotiation successful
0808 02:18:19.264424 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":159,"threads_started":1}
I20260808 02:18:19.265466 2205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.252327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52354 (local address 127.24.79.253:38013)
0808 02:18:19.252858 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:19.252877 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:19.252920 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:18:19.253679 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:19.253702 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:19.253917 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:19.254071 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:19.254937 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.255951 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.256930 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.257622 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.257748 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.260994 (+ 3246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:19.261000 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:19.261014 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:19.261038 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:19.262731 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.263270 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.263276 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.263277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.263356 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.263665 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.263669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.263671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.263941 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:19.264205 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:19.264361 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 02:18:19.264508 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"spinlock_wait_cycles":55552,"thread_start_us":206,"threads_started":1}
I20260808 02:18:19.267112 2194 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:18:19 dist-test-slave-1vmg krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155499, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.253@KRBTEST.COM for oryx/127.24.79.253@KRBTEST.COM
I20260808 02:18:19.275868 2205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.267722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52360 (local address 127.24.79.253:38013)
0808 02:18:19.268046 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:19.268049 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:19.268059 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:18:19.268085 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:19.268087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:19.268119 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:19.268182 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:19.268602 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.269454 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.270215 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.270454 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.270541 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.272712 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:19.272716 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:19.272719 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:19.272742 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:19.274151 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.274564 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.274568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.274570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.274649 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.274892 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.274895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.274897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.275182 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:19.275278 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:19.275523 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 02:18:19.275729 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":103,"server-negotiator.queue_time_us":267}
I20260808 02:18:19.275868 2203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.267650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:38013 (local address 127.0.0.1:52360)
0808 02:18:19.267755 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:19.267792 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:19.267869 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:19.268214 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:19.268217 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:19.268383 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:19.268486 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.268491 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.269567 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.269570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.270106 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.270114 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.270556 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.270560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.270691 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.271213 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:19.271225 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:19.272566 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:19.274274 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.274278 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.274280 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.274472 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.274731 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.274734 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.274736 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.274811 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.275280 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:19.275283 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:19.275373 (+ 90us) client_negotiation.cc:769] Sending connection context
0808 02:18:19.275463 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 02:18:19.275660 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":28}
I20260808 02:18:19.279476 2194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2: Bootstrap starting.
I20260808 02:18:19.282105 2194 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:19.282889 2194 log.cc:824] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:19.285211 2194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2: No bootstrap required, opened a new log
I20260808 02:18:19.288542 2194 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } }
I20260808 02:18:19.288784 2194 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:19.288820 2194 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b0b6a0e3064489a90c4859f80edc3f2, State: Initialized, Role: FOLLOWER
I20260808 02:18:19.289254 2194 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } }
I20260808 02:18:19.289786 2207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } } }
I20260808 02:18:19.289924 2207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:18:19.290568 2194 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 02:18:19.296013 2229 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:18:19.296136 2229 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:18:19.298652 2093 master_runner.cc:428] Detected that this master 5b0b6a0e3064489a90c4859f80edc3f2 is joining an existing cluster
I20260808 02:18:19.298730 2093 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.79.253:38013
I20260808 02:18:19.311043 1468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.301233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41090 (local address 127.24.79.254:45167)
0808 02:18:19.301372 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:19.301376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:19.302204 (+ 828us) server_negotiation.cc:407] Connection header received
0808 02:18:19.302368 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:19.302372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:19.302442 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:19.302566 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:19.303178 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.303933 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.305227 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.305566 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.305700 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.307288 (+ 1588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:19.307291 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:19.307294 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:19.307321 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:19.309076 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.309692 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.309695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.309697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.309788 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.310068 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.310072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.310073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.310287 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:19.310375 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:19.310701 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 02:18:19.310838 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 02:18:19.311043 2205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.301794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52370 (local address 127.24.79.253:38013)
0808 02:18:19.301910 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:19.301914 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:19.302560 (+ 646us) server_negotiation.cc:407] Connection header received
0808 02:18:19.302732 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:19.302735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:19.302779 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:19.302864 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:19.303436 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.304135 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.305332 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.305651 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.305732 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.307432 (+ 1700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:19.307436 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:19.307438 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:19.307462 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:19.309189 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.309632 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.309635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.309637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.309725 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.309994 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.309997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.309999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.310243 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:19.310351 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:19.310657 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 02:18:19.310842 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":85,"server-negotiator.queue_time_us":41}
I20260808 02:18:19.311071 2235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.301698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:38013 (local address 127.0.0.1:52370)
0808 02:18:19.302483 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:19.302496 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:19.302628 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:19.302906 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:19.302931 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:19.303171 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:19.303316 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.303324 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.304260 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.304264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.305204 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.305216 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.306226 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.306230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.306325 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.306763 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:19.306778 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:19.307289 (+ 511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:19.309271 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.309274 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.309276 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.309533 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.309822 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.309825 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.309827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.309909 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.310339 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:19.310343 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:19.310458 (+ 115us) client_negotiation.cc:769] Sending connection context
0808 02:18:19.310590 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 02:18:19.310777 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"mutex_wait_us":42,"spinlock_wait_cycles":17152,"thread_start_us":104,"threads_started":1}
I20260808 02:18:19.311643 2236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.301125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.0.0.1:41090)
0808 02:18:19.302072 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:19.302087 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:19.302216 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:19.302610 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:19.302613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:19.302897 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:19.303049 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.303056 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.304069 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.304073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.305093 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.305105 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.305726 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.305730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.305950 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.306547 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:19.306565 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:19.307123 (+ 558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:19.309227 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.309231 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.309233 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.309585 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.309875 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.309877 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.309879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.309974 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.310419 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:19.310422 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:19.310518 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 02:18:19.311313 (+ 795us) client_negotiation.cc:240] Negotiation successful
0808 02:18:19.311470 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":847,"mutex_wait_us":5,"thread_start_us":139,"threads_started":1}
Aug 08 02:18:19 dist-test-slave-1vmg krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155498, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.254@KRBTEST.COM for oryx/127.24.79.253@KRBTEST.COM
I20260808 02:18:19.328131 2205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.317406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52380 (local address 127.24.79.253:38013)
0808 02:18:19.317525 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:19.317528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:19.317904 (+ 376us) server_negotiation.cc:407] Connection header received
0808 02:18:19.318105 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:19.318108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:19.318166 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:19.318235 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:19.319054 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.319727 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.321076 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.321376 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.321482 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.324097 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:19.324102 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:19.324105 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:19.324134 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:19.325707 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.326386 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.326390 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.326392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.326492 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.326856 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.326860 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.326862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.327156 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:19.327238 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:19.327798 (+ 560us) server_negotiation.cc:299] Negotiation successful
0808 02:18:19.327945 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 02:18:19.329107 2238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.317379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:38013 (local address 127.0.0.1:52380)
0808 02:18:19.317777 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:19.317824 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:19.317969 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:19.318292 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:19.318298 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:19.318746 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:19.318898 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.318907 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.319869 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.319873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.320937 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.320951 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.321643 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.321647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.321744 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.322330 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:19.322343 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:19.323937 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:19.325862 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.325876 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.325886 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.326254 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.326623 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.326626 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.326629 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.326746 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.327335 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:19.327343 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:19.327510 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 02:18:19.328600 (+ 1090us) client_negotiation.cc:240] Negotiation successful
0808 02:18:19.328909 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":118,"threads_started":1}
WARNING: no policy specified for oryx/127.24.79.252@KRBTEST.COM; defaulting to no policy
I20260808 02:18:19.332140 1405 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.79.253:38013: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2"
member_type: NON_VOTER
last_known_addr {
host: "127.24.79.253"
port: 38013
}
attrs {
promote: true
}
}
dest_uuid: "aa78ed5a894742328b0e31dda1f3499a"
cas_config_opid_index: -1
I20260808 02:18:19.333143 1405 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: NON_VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.79.252@KRBTEST.COM" created.
I20260808 02:18:19.335486 2245 raft_consensus.cc:2932] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 5b0b6a0e3064489a90c4859f80edc3f2 (127.24.79.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: NON_VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: true } } }
I20260808 02:18:19.337037 2151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:19.338644 2151 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 1 FOLLOWER]: Refusing update from remote peer aa78ed5a894742328b0e31dda1f3499a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 02:18:19.339787 2246 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: NON_VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:18:19.340581 2245 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.79.253:38013
I20260808 02:18:19.341193 2245 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "aa78ed5a894742328b0e31dda1f3499a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: NON_VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: true } } }
I20260808 02:18:19.341262 2093 master.cc:537] Master@127.24.79.253:38013 shutting down...
I20260808 02:18:19.341426 2245 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [sys.catalog]: This master's current role is: LEADER
W20260808 02:18:19.342549 1390 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 02:18:19.343580 1390 consensus_peers.cc:487] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a -> Peer 5b0b6a0e3064489a90c4859f80edc3f2 (127.24.79.253:38013): Couldn't send request to peer 5b0b6a0e3064489a90c4859f80edc3f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 02:18:19.346391 2093 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:18:19.346736 2093 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:18:19.346858 2093 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.79.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.252.keytab.
Entry for principal oryx/127.24.79.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.252.keytab.
I20260808 02:18:19.359812 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.252:44521
--webserver_interface=127.24.79.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.252.keytab
--principal=oryx/127.24.79.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:45167,127.24.79.253:38013,127.24.79.252:44521
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:18:19.362705 2093 master.cc:552] Master@127.24.79.253:38013 shutdown complete.
I20260808 02:18:19.362777 2093 master_runner.cc:305] Clearing existing system tablet
I20260808 02:18:19.363081 2093 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:19.363147 2093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:19.364341 2093 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:18:19.365211 2255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:19.365468 2093 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:19.365603 2093 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:18:19.365648 2093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/wal
uuid: "5b0b6a0e3064489a90c4859f80edc3f2"
format_stamp: "Formatted at 2026-08-08 02:18:19 on dist-test-slave-1vmg"
server_key: "2d23d132a5c649b654225028dcec27e2"
server_key_iv: "8370b5e3f6f568bfacaa88acd7256e5e"
server_key_version: "encryptionkey@0"
I20260808 02:18:19.367898 2093 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:18:19.370074 2093 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 02:18:19.370182 2093 log.cc:1196] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 02:18:19.370450 2093 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2: Deleting consensus metadata
I20260808 02:18:19.370645 2093 master_runner.cc:315] Copying system tablet from 127.24.79.254:45167
I20260808 02:18:19.372131 2093 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2: tablet copy: Beginning tablet copy session from remote peer at address 127.24.79.254:45167
I20260808 02:18:19.380682 1468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.372933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41104 (local address 127.24.79.254:45167)
0808 02:18:19.373094 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:19.373099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:19.373337 (+ 238us) server_negotiation.cc:407] Connection header received
0808 02:18:19.373475 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:19.373478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:19.373532 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:19.373618 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:19.374165 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.374708 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.375691 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.376000 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.376097 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.377434 (+ 1337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:19.377438 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:19.377441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:19.377469 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:19.378899 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.379396 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.379400 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.379402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.379490 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.379774 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.379778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.379780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.379990 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:19.380086 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:19.380384 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 02:18:19.380503 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:18:19.380693 2262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.372825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.0.0.1:41104)
0808 02:18:19.373227 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:19.373241 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:19.373371 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:19.373676 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:19.373679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:19.373887 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:19.374021 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.374027 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.374831 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.374834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.375526 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.375539 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.376268 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.376271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.376352 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.376888 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:19.376901 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:19.377271 (+ 370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:19.379063 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.379073 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.379076 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.379295 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.379583 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.379586 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.379588 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.379677 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.380110 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:19.380113 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:19.380217 (+ 104us) client_negotiation.cc:769] Sending connection context
0808 02:18:19.380310 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 02:18:19.380481 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":107,"threads_started":1}
I20260808 02:18:19.381807 1438 tablet_copy_service.cc:140] P aa78ed5a894742328b0e31dda1f3499a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5b0b6a0e3064489a90c4859f80edc3f2 ({username='oryx', principal='oryx/127.24.79.253@KRBTEST.COM'} at 127.0.0.1:41104)
I20260808 02:18:19.382027 1438 tablet_copy_service.cc:161] P aa78ed5a894742328b0e31dda1f3499a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5b0b6a0e3064489a90c4859f80edc3f2 at {username='oryx', principal='oryx/127.24.79.253@KRBTEST.COM'} at 127.0.0.1:41104: session id = 5b0b6a0e3064489a90c4859f80edc3f2-00000000000000000000000000000000
I20260808 02:18:19.383872 1438 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 02:18:19.386149 2093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 02:18:19.388517 2093 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2: tablet copy: Starting download of 0 data blocks...
I20260808 02:18:19.388756 2093 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2: tablet copy: Starting download of 1 WAL segments...
I20260808 02:18:19.390841 2093 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 02:18:19.392933 1438 tablet_copy_service.cc:342] P aa78ed5a894742328b0e31dda1f3499a: Request end of tablet copy session 5b0b6a0e3064489a90c4859f80edc3f2-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.79.253@KRBTEST.COM'} at 127.0.0.1:41104
I20260808 02:18:19.393097 1438 tablet_copy_service.cc:433] P aa78ed5a894742328b0e31dda1f3499a: ending tablet copy session 5b0b6a0e3064489a90c4859f80edc3f2-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5b0b6a0e3064489a90c4859f80edc3f2
I20260808 02:18:19.401005 2093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:19.403010 2263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:19.403014 2266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:19.403605 2093 server_base.cc:1034] running on GCE node
W20260808 02:18:19.407982 2264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:19.408370 2093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:19.408774 2093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:19.409935 2093 hybrid_clock.cc:648] HybridClock initialized: now 1786155499409923 us; error 40 us; skew 500 ppm
Aug 08 02:18:19 dist-test-slave-1vmg krb5kdc[1339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155499, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:19.411572 2093 init.cc:377] Logged in from keytab as oryx/127.24.79.253@KRBTEST.COM (short username oryx)
I20260808 02:18:19.412163 2093 webserver.cc:460] Webserver started at http://127.24.79.253:36595/ using document root <none> and password file <none>
I20260808 02:18:19.412366 2093 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:19.412406 2093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:19.413789 2093 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:18:19.414769 2276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:19.415132 2093 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:18:19.415222 2093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/wal
uuid: "5b0b6a0e3064489a90c4859f80edc3f2"
format_stamp: "Formatted at 2026-08-08 02:18:19 on dist-test-slave-1vmg"
server_key: "2d23d132a5c649b654225028dcec27e2"
server_key_iv: "8370b5e3f6f568bfacaa88acd7256e5e"
server_key_version: "encryptionkey@0"
I20260808 02:18:19.415315 2093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:19.445335 2093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:19.447605 2093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:19.452719 2355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.253:38013 every 8 connection(s)
I20260808 02:18:19.452714 2093 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.253:38013
I20260808 02:18:19.453393 2093 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-1/data/info.pb
I20260808 02:18:19.455084 2358 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:18:19.456332 2358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2: Bootstrap starting.
I20260808 02:18:19.467176 2358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:18:19.467621 2358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2: Bootstrap complete.
I20260808 02:18:19.468251 2358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: NON_VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: true } }
I20260808 02:18:19.468381 2358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b0b6a0e3064489a90c4859f80edc3f2, State: Initialized, Role: LEARNER
I20260808 02:18:19.468490 2358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: NON_VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: true } }
I20260808 02:18:19.468853 2363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: NON_VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: true } } }
I20260808 02:18:19.469000 2358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:18:19.469033 2363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [sys.catalog]: This master's current role is: LEARNER
I20260808 02:18:19.472128 2376 catalog_manager.cc:1259] Loaded cluster ID: 0a4525e343e5431e9aa0df4f93800b9c
I20260808 02:18:19.472302 2376 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2: loading cluster ID for follower catalog manager: success
I20260808 02:18:19.474040 2376 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2: acquiring CA information for follower catalog manager: success
I20260808 02:18:19.475440 2376 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:18:19.484381 1689 heartbeater.cc:503] Master 127.24.79.254:45167 was elected leader, sending a full tablet report...
W20260808 02:18:19.523940 2253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:19.524271 2253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:19.524317 2253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:19.528791 2253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:18:19.528895 2253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:19.528915 2253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:19.528934 2253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:18:19.528947 2253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:18:19.535409 2253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:45167,127.24.79.253:38013,127.24.79.252:44521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.252.keytab
--principal=oryx/127.24.79.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.252:44521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.79.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:19.536742 2253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:19.537982 2253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:19.548182 2390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:19.548139 2386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:19.548209 2387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:19.548645 2253 server_base.cc:1034] running on GCE node
I20260808 02:18:19.549369 2253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:19.550726 2253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:19.551944 2253 hybrid_clock.cc:648] HybridClock initialized: now 1786155499551918 us; error 45 us; skew 500 ppm
Aug 08 02:18:19 dist-test-slave-1vmg krb5kdc[1339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155499, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:19.555295 2253 init.cc:377] Logged in from keytab as oryx/127.24.79.252@KRBTEST.COM (short username oryx)
I20260808 02:18:19.556457 2253 webserver.cc:460] Webserver started at http://127.24.79.252:34019/ using document root <none> and password file <none>
I20260808 02:18:19.557082 2253 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:19.557168 2253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:19.557408 2253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:19.559844 2253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/data/instance:
uuid: "44c68eddae7e4a04b957b0ead4437ae8"
format_stamp: "Formatted at 2026-08-08 02:18:19 on dist-test-slave-1vmg"
server_key: "fe272037ebcc551dfd4910c6d8dddedd"
server_key_iv: "13730915abac62b5e188d96bb34025e5"
server_key_version: "encryptionkey@0"
I20260808 02:18:19.560503 2253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/wal/instance:
uuid: "44c68eddae7e4a04b957b0ead4437ae8"
format_stamp: "Formatted at 2026-08-08 02:18:19 on dist-test-slave-1vmg"
server_key: "fe272037ebcc551dfd4910c6d8dddedd"
server_key_iv: "13730915abac62b5e188d96bb34025e5"
server_key_version: "encryptionkey@0"
I20260808 02:18:19.565009 2253 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:18:19.567977 2399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:19.569470 2253 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:19.569649 2253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/wal
uuid: "44c68eddae7e4a04b957b0ead4437ae8"
format_stamp: "Formatted at 2026-08-08 02:18:19 on dist-test-slave-1vmg"
server_key: "fe272037ebcc551dfd4910c6d8dddedd"
server_key_iv: "13730915abac62b5e188d96bb34025e5"
server_key_version: "encryptionkey@0"
I20260808 02:18:19.569785 2253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:19.582176 2253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:19.586143 2253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:19.586352 2253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:19.594163 2253 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.252:44521
I20260808 02:18:19.594475 2479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.252:44521 every 8 connection(s)
I20260808 02:18:19.595360 2253 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/data/info.pb
I20260808 02:18:19.598510 2480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:19.599964 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 2253
I20260808 02:18:19.603468 2480 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:18:19 dist-test-slave-1vmg krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.79.252@KRBTEST.COM
Aug 08 02:18:19 dist-test-slave-1vmg krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155499, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.252@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM
I20260808 02:18:19.616811 1468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.605767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41110 (local address 127.24.79.254:45167)
0808 02:18:19.605912 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:19.605916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:19.606126 (+ 210us) server_negotiation.cc:407] Connection header received
0808 02:18:19.606275 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:19.606279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:19.606330 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:19.606424 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:19.607426 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.608164 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.609458 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.609684 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.609781 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.612689 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:19.612693 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:19.612695 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:19.612726 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:19.614433 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.615077 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.615080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.615081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.615167 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.615562 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.615566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.615568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.615824 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:19.615925 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:19.616413 (+ 488us) server_negotiation.cc:299] Negotiation successful
0808 02:18:19.616639 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 02:18:19.617410 2484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.605534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.0.0.1:41110)
0808 02:18:19.605987 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:19.606043 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:19.606180 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:19.606498 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:19.606504 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:19.607069 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:19.607241 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.607249 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.608349 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.608352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.609304 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.609319 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.609881 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.609886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.610014 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.610646 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:19.610667 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:19.612506 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:19.614549 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.614560 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.614568 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.614971 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.615272 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.615276 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.615278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.615378 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.615943 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:19.615951 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:19.616095 (+ 144us) client_negotiation.cc:769] Sending connection context
0808 02:18:19.616251 (+ 156us) client_negotiation.cc:240] Negotiation successful
0808 02:18:19.616466 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":141,"threads_started":1}
I20260808 02:18:19.617421 2482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.601762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39776 (local address 127.24.79.252:44521)
0808 02:18:19.602306 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:19.602329 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:19.602382 (+ 53us) server_negotiation.cc:407] Connection header received
0808 02:18:19.603281 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:19.603311 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:19.603814 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:19.604280 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:19.605077 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.606320 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.607613 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.608405 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.608543 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.611685 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:19.611693 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:19.611708 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:19.611735 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:19.613777 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.614390 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.614397 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.614405 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.614546 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.614858 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.614861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.614862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.615440 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:19.615789 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:19.616002 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 02:18:19.616374 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":158,"threads_started":1}
I20260808 02:18:19.619166 2480 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:18:19 dist-test-slave-1vmg krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155499, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.252@KRBTEST.COM for oryx/127.24.79.253@KRBTEST.COM
I20260808 02:18:19.628257 2484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.619743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:38013 (local address 127.0.0.1:52396)
0808 02:18:19.619883 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:19.619895 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:19.619983 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:19.620405 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:19.620407 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:19.620561 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:19.620671 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.620676 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.621705 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.621715 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.622259 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.622267 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.622773 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.622776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.622853 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.623436 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:19.623449 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:19.624863 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:19.626704 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.626707 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.626709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.626984 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.627276 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.627278 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.627279 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.627367 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.627782 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:19.627784 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:19.627886 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 02:18:19.627974 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 02:18:19.628103 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 02:18:19.628271 2485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.619879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52396 (local address 127.24.79.253:38013)
0808 02:18:19.620195 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:19.620199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:19.620211 (+ 12us) server_negotiation.cc:407] Connection header received
0808 02:18:19.620251 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:19.620253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:19.620293 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:19.620368 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:19.620803 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.621563 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.622416 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.622670 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.622768 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.625045 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:19.625049 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:19.625051 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:19.625079 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:19.626544 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.627081 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.627083 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.627085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.627189 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.627449 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.627451 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.627452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.627687 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:19.627777 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:19.628011 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 02:18:19.628129 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":118,"threads_started":1}
I20260808 02:18:19.629163 2480 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:18:19 dist-test-slave-1vmg krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155499, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.252@KRBTEST.COM for oryx/127.24.79.252@KRBTEST.COM
I20260808 02:18:19.637897 2484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.629723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:44521 (local address 127.0.0.1:39790)
0808 02:18:19.629837 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:19.629851 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:19.629927 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:19.630089 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:19.630091 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:19.630251 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:19.630334 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.630339 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.631582 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.631585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.632124 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.632133 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.632661 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.632664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.632738 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.633291 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:19.633304 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:19.634687 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:19.636293 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.636296 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.636298 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.636504 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.636879 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.636882 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.636883 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.636984 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.637416 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:19.637419 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:19.637516 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 02:18:19.637614 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 02:18:19.637757 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 02:18:19.637956 2482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.629806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39790 (local address 127.24.79.252:44521)
0808 02:18:19.629891 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:19.629894 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:19.629904 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:18:19.629985 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:19.629987 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:19.630022 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:19.630079 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:19.630418 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.631432 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.632306 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.632549 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.632639 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.634820 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:19.634823 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:19.634825 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:19.634845 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:19.636172 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.636639 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.636643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.636644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.636748 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.637082 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.637084 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.637085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.637307 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:19.637388 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:19.637663 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 02:18:19.637826 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":28}
I20260808 02:18:19.641599 2480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8: Bootstrap starting.
I20260808 02:18:19.644181 2480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:19.644960 2480 log.cc:824] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:19.647347 2480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8: No bootstrap required, opened a new log
I20260808 02:18:19.650729 2480 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } }
I20260808 02:18:19.651042 2480 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:19.651106 2480 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44c68eddae7e4a04b957b0ead4437ae8, State: Initialized, Role: FOLLOWER
I20260808 02:18:19.651652 2480 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } }
I20260808 02:18:19.651877 2487 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:18:19.652061 2487 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } }
I20260808 02:18:19.652536 2488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } } }
I20260808 02:18:19.652689 2488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:18:19.653160 2487 leader_election.cc:288] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa78ed5a894742328b0e31dda1f3499a (127.24.79.254:45167), 5b0b6a0e3064489a90c4859f80edc3f2 (127.24.79.253:38013)
I20260808 02:18:19.653214 2480 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:18:19.653836 1425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44c68eddae7e4a04b957b0ead4437ae8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa78ed5a894742328b0e31dda1f3499a" is_pre_election: true
I20260808 02:18:19.654004 2311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44c68eddae7e4a04b957b0ead4437ae8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" is_pre_election: true
I20260808 02:18:19.654232 2311 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 1 LEARNER]: Handling vote request from an unknown peer 44c68eddae7e4a04b957b0ead4437ae8
I20260808 02:18:19.654403 2311 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 44c68eddae7e4a04b957b0ead4437ae8 for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260808 02:18:19.654803 2405 leader_election.cc:302] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 44c68eddae7e4a04b957b0ead4437ae8; no voters: 5b0b6a0e3064489a90c4859f80edc3f2, aa78ed5a894742328b0e31dda1f3499a
I20260808 02:18:19.655362 2487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:19.656469 2487 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 02:18:19.660765 2506 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:18:19.660872 2506 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:18:19.663932 2253 master_runner.cc:428] Detected that this master 44c68eddae7e4a04b957b0ead4437ae8 is joining an existing cluster
I20260808 02:18:19.664000 2253 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.79.252:44521
I20260808 02:18:19.671698 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 1503
I20260808 02:18:19.713260 2485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.666886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52400 (local address 127.24.79.253:38013)
0808 02:18:19.667034 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:19.667039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:19.667149 (+ 110us) server_negotiation.cc:407] Connection header received
0808 02:18:19.667345 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:19.667348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:19.667395 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:19.667443 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:19.668444 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.669318 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.670385 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.670661 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.670756 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.709479 (+ 38723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:19.709484 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:19.709487 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:19.709523 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:19.711254 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.711813 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.711818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.711820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.711919 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.712226 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.712229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.712231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.712467 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:19.712536 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:19.712931 (+ 395us) server_negotiation.cc:299] Negotiation successful
0808 02:18:19.713080 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 02:18:19.713871 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.666760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:38013 (local address 127.0.0.1:52400)
0808 02:18:19.667069 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:19.667079 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:19.667225 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:19.667577 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:19.667579 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:19.667919 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:19.668308 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.668318 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.669452 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.669455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.670252 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.670264 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.670950 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.670954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.671041 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.700563 (+ 29522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:19.700585 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:19.709262 (+ 8677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:19.711394 (+ 2132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.711398 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.711400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.711675 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.712034 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.712037 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.712039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.712130 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.712611 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:19.712614 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:19.712717 (+ 103us) client_negotiation.cc:769] Sending connection context
0808 02:18:19.713500 (+ 783us) client_negotiation.cc:240] Negotiation successful
0808 02:18:19.713665 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"spinlock_wait_cycles":8704,"thread_start_us":90,"threads_started":1}
I20260808 02:18:19.717357 1468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.666388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41122 (local address 127.24.79.254:45167)
0808 02:18:19.666533 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:19.666539 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:19.667180 (+ 641us) server_negotiation.cc:407] Connection header received
0808 02:18:19.667367 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:19.667370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:19.667424 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:19.667476 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:19.668269 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.668849 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.669878 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.670258 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.670359 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.708708 (+ 38349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:19.708714 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:19.708717 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:19.708763 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:19.712487 (+ 3724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.715665 (+ 3178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.715670 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.715672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.715808 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.716192 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.716196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.716198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.716442 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:19.716526 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:19.717030 (+ 504us) server_negotiation.cc:299] Negotiation successful
0808 02:18:19.717196 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:18:19.717983 2515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.666299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.0.0.1:41122)
0808 02:18:19.667086 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:19.667100 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:19.667253 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:19.667630 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:19.667633 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:19.667987 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:19.668117 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.668125 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.668953 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.668956 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.669748 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.669758 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.670515 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.670520 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.670615 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.671282 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:19.671297 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:19.708393 (+ 37096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:19.714556 (+ 6163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.714559 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.714560 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.714854 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.715987 (+ 1133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.715989 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.715991 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.716084 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.716599 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:19.716601 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:19.716794 (+ 193us) client_negotiation.cc:769] Sending connection context
0808 02:18:19.717539 (+ 745us) client_negotiation.cc:240] Negotiation successful
0808 02:18:19.717799 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":126,"threads_started":1}
I20260808 02:18:19.718904 2482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.666448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39804 (local address 127.24.79.252:44521)
0808 02:18:19.666591 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:19.666595 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:19.667143 (+ 548us) server_negotiation.cc:407] Connection header received
0808 02:18:19.667320 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:19.667323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:19.667361 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:19.667407 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:19.709537 (+ 42130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.710894 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.713680 (+ 2786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.713986 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.714066 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.715592 (+ 1526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:19.715596 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:19.715599 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:19.715625 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:19.717327 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.717769 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.717772 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.717774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.717846 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.718072 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.718074 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.718076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.718264 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:19.718346 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:19.718633 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 02:18:19.718758 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 02:18:19.719129 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:46067
--local_ip_for_outbound_sockets=127.24.79.193
--tserver_master_addrs=127.24.79.254:45167
--webserver_port=33151
--webserver_interface=127.24.79.193
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.193.keytab
--principal=oryx/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.79.254:45167,127.24.79.253:38013
--tserver_master_addrs=127.24.79.254:45167,127.24.79.253:38013,127.24.79.252:44521 with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:18:19.719372 2516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.666442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:44521 (local address 127.0.0.1:39804)
0808 02:18:19.667063 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:19.667079 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:19.667224 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:19.667542 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:19.667545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:19.708440 (+ 40895us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:19.708651 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.708663 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.712653 (+ 3990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.712657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.713534 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.713549 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.714215 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.714219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.714311 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.715026 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:19.715039 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:19.715448 (+ 409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:19.717436 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.717439 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.717441 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.717663 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.717927 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.717930 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.717931 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.717999 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.718389 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:19.718391 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:19.718476 (+ 85us) client_negotiation.cc:769] Sending connection context
0808 02:18:19.719074 (+ 598us) client_negotiation.cc:240] Negotiation successful
0808 02:18:19.719215 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"spinlock_wait_cycles":768,"thread_start_us":93,"threads_started":1}
I20260808 02:18:19.720921 1895 heartbeater.cc:503] Master 127.24.79.254:45167 was elected leader, sending a full tablet report...
Aug 08 02:18:19 dist-test-slave-1vmg krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155498, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.254@KRBTEST.COM for oryx/127.24.79.252@KRBTEST.COM
I20260808 02:18:19.732577 2238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.722632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:44521 (local address 127.0.0.1:39808)
0808 02:18:19.722895 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:19.722909 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:19.723025 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:19.723347 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:19.723350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:19.723560 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:19.723683 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.723690 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.724776 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.724780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.725642 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.725655 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.726243 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.726247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.726356 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.726988 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:19.727009 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:19.728721 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:19.730782 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.730786 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.730788 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.731110 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.731436 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.731439 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.731441 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.731536 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.732002 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:19.732005 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:19.732122 (+ 117us) client_negotiation.cc:769] Sending connection context
0808 02:18:19.732221 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 02:18:19.732387 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188}
I20260808 02:18:19.732903 2482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.722681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39808 (local address 127.24.79.252:44521)
0808 02:18:19.722800 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:19.722805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:19.722978 (+ 173us) server_negotiation.cc:407] Connection header received
0808 02:18:19.723150 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:19.723153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:19.723210 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:19.723805 (+ 595us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:19.723912 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.724664 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.725776 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.726113 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.726218 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.728923 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:19.728927 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:19.728930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:19.728963 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:19.730639 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.731232 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.731236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.731238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.731340 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.731645 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.731648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.731650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.731905 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:19.732011 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:19.732313 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 02:18:19.732449 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 02:18:19.734115 1405 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.79.252:44521: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8"
member_type: NON_VOTER
last_known_addr {
host: "127.24.79.252"
port: 44521
}
attrs {
promote: true
}
}
dest_uuid: "aa78ed5a894742328b0e31dda1f3499a"
cas_config_opid_index: 5
I20260808 02:18:19.734834 1405 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: NON_VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: true } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: NON_VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: true } }
I20260808 02:18:19.736027 2245 raft_consensus.cc:2932] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 44c68eddae7e4a04b957b0ead4437ae8 (127.24.79.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: NON_VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: true } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: NON_VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: true } } }
I20260808 02:18:19.737490 2441 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 1 FOLLOWER]: Refusing update from remote peer aa78ed5a894742328b0e31dda1f3499a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 02:18:19.737569 2245 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.79.252:44521
I20260808 02:18:19.737780 2245 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "aa78ed5a894742328b0e31dda1f3499a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: NON_VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: true } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: NON_VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: true } } }
I20260808 02:18:19.737884 2245 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:19.738150 2249 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [LEADER]: Connected to new peer: Peer: permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: NON_VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:18:19.743218 2441 raft_consensus.cc:858] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: NON_VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: NON_VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: true } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: NON_VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: true } }
I20260808 02:18:19.743813 2253 master.cc:537] Master@127.24.79.252:44521 shutting down...
W20260808 02:18:19.745414 2441 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: NON_VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: NON_VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: true } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: NON_VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: true } }
I20260808 02:18:19.751371 2441 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 5b0b6a0e3064489a90c4859f80edc3f2 (127.24.79.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: NON_VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: true } } }
I20260808 02:18:19.752436 2238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.736220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:38013 (local address 127.0.0.1:52412)
0808 02:18:19.747339 (+ 11119us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:19.747362 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:19.747594 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:19.748091 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:19.748094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:19.748117 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:19.748229 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.748235 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.749335 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.749339 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.750598 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.750611 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.751936 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.751940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.752043 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.752108 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 02:18:19.752197 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 02:18:19.752288 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10996}
I20260808 02:18:19.752463 2485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.736485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52412 (local address 127.24.79.253:38013)
0808 02:18:19.739295 (+ 2810us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:19.739301 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:19.747472 (+ 8171us) server_negotiation.cc:407] Connection header received
0808 02:18:19.747738 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:19.747742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:19.747813 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:19.747889 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:19.748369 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.749166 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.750766 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.751462 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.751580 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.751781 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:19.752237 (+ 456us) server_negotiation.cc:299] Negotiation successful
0808 02:18:19.752334 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2696}
I20260808 02:18:19.754814 2488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aa78ed5a894742328b0e31dda1f3499a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: NON_VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: true } } }
I20260808 02:18:19.755046 2488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 02:18:19.755175 2488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aa78ed5a894742328b0e31dda1f3499a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: NON_VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: true } } }
I20260808 02:18:19.755239 2488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 02:18:19.755415 2487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa78ed5a894742328b0e31dda1f3499a. Latest consensus state: current_term: 1 leader_uuid: "aa78ed5a894742328b0e31dda1f3499a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: NON_VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: true } } }
I20260808 02:18:19.755479 2487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 02:18:19.756438 2311 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 1 LEARNER]: Refusing update from remote peer aa78ed5a894742328b0e31dda1f3499a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 02:18:19.757036 2249 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: NON_VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 02:18:19.759438 2311 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 44c68eddae7e4a04b957b0ead4437ae8 (127.24.79.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: NON_VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: true } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: NON_VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: true } } }
I20260808 02:18:19.760797 2520 mvcc.cc:205] Tried to move back new op lower bound from 7316092920573018112 to 7316092920410533888. Current Snapshot: MvccSnapshot[applied={T|T < 7316092920573018112 or (T in {7316092920573018112})}]
I20260808 02:18:19.761551 2253 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260808 02:18:19.761570 2249 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aa78ed5a894742328b0e31dda1f3499a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: NON_VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: true } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: NON_VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: true } } }
I20260808 02:18:19.761730 2249 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:19.761871 2253 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260808 02:18:19.761972 2253 tablet_replica.cc:330] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8: stopping tablet replica
W20260808 02:18:19.763188 1387 consensus_peers.cc:487] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a -> Peer 44c68eddae7e4a04b957b0ead4437ae8 (127.24.79.252:44521): Couldn't send request to peer 44c68eddae7e4a04b957b0ead4437ae8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 02:18:19.765815 2363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa78ed5a894742328b0e31dda1f3499a. Latest consensus state: current_term: 1 leader_uuid: "aa78ed5a894742328b0e31dda1f3499a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: NON_VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: true } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: NON_VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: true } } }
I20260808 02:18:19.765986 2363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [sys.catalog]: This master's current role is: LEARNER
I20260808 02:18:19.766091 2363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aa78ed5a894742328b0e31dda1f3499a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: NON_VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: true } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: NON_VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: true } } }
I20260808 02:18:19.766160 2363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [sys.catalog]: This master's current role is: LEARNER
I20260808 02:18:19.766428 2245 raft_consensus.cc:1050] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a: attempting to promote NON_VOTER 5b0b6a0e3064489a90c4859f80edc3f2 to VOTER
I20260808 02:18:19.767056 2245 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: NON_VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: true } }
I20260808 02:18:19.768555 2238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.768020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:44521 (local address 127.0.0.1:39818)
0808 02:18:19.768350 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:19.768439 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238}
I20260808 02:18:19.768939 2311 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 1 LEARNER]: Refusing update from remote peer aa78ed5a894742328b0e31dda1f3499a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 02:18:19.769642 2238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.769325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:44521 (local address 127.0.0.1:39824)
0808 02:18:19.769501 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:19.769558 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260808 02:18:19.769742 2248 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260808 02:18:19.771533 1387 consensus_peers.cc:487] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a -> Peer 44c68eddae7e4a04b957b0ead4437ae8 (127.24.79.252:44521): Couldn't send request to peer 44c68eddae7e4a04b957b0ead4437ae8. Status: Network error: Client connection negotiation failed: client connection to 127.24.79.252:44521: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 02:18:19.771961 2253 master.cc:552] Master@127.24.79.252:44521 shutdown complete.
I20260808 02:18:19.772058 2253 master_runner.cc:305] Clearing existing system tablet
I20260808 02:18:19.772359 2253 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:19.772476 2253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:19.772831 2248 raft_consensus.cc:2932] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5b0b6a0e3064489a90c4859f80edc3f2 (127.24.79.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: NON_VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: true } } }
I20260808 02:18:19.774542 2248 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "aa78ed5a894742328b0e31dda1f3499a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: NON_VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: true } } }
I20260808 02:18:19.774690 2248 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:19.774829 2248 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aa78ed5a894742328b0e31dda1f3499a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: NON_VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: true } } }
I20260808 02:18:19.775257 2248 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:19.775700 2311 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5b0b6a0e3064489a90c4859f80edc3f2 (127.24.79.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: NON_VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: true } } }
I20260808 02:18:19.777367 2363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aa78ed5a894742328b0e31dda1f3499a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: NON_VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: true } } }
I20260808 02:18:19.777499 2363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:18:19.778460 2253 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260808 02:18:19.779620 2534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:19.779865 2253 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:19.780062 2253 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 02:18:19.780122 2253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/wal
uuid: "44c68eddae7e4a04b957b0ead4437ae8"
format_stamp: "Formatted at 2026-08-08 02:18:19 on dist-test-slave-1vmg"
server_key: "fe272037ebcc551dfd4910c6d8dddedd"
server_key_iv: "13730915abac62b5e188d96bb34025e5"
server_key_version: "encryptionkey@0"
I20260808 02:18:19.782168 2253 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:18:19.784880 2253 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 02:18:19.785038 2253 log.cc:1196] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 02:18:19.785386 2253 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8: Deleting consensus metadata
I20260808 02:18:19.785598 2253 master_runner.cc:315] Copying system tablet from 127.24.79.254:45167
I20260808 02:18:19.786965 2253 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8: tablet copy: Beginning tablet copy session from remote peer at address 127.24.79.254:45167
I20260808 02:18:19.797068 1468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.787792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41128 (local address 127.24.79.254:45167)
0808 02:18:19.787941 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:19.787946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:19.788227 (+ 281us) server_negotiation.cc:407] Connection header received
0808 02:18:19.788386 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:19.788389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:19.788452 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:19.788536 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:19.789099 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.789924 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.791252 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.791601 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.791713 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.793213 (+ 1500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:19.793218 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:19.793222 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:19.793255 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:19.795098 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.795683 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.795687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.795688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.795799 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.796076 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.796080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.796081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.796338 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:19.796446 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:19.796763 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 02:18:19.796897 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:18:19.797080 2544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.787686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.0.0.1:41128)
0808 02:18:19.788094 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:19.788111 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:19.788258 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:19.788578 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:19.788581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:19.788828 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:19.788959 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.788966 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.790083 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.790087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.791114 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.791127 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.791732 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.791735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.791851 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.792486 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:19.792502 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:19.792983 (+ 481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:19.795240 (+ 2257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.795244 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.795247 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.795569 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.795889 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.795892 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.795895 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.795987 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.796448 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:19.796451 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:19.796578 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 02:18:19.796694 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 02:18:19.796888 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":103,"threads_started":1}
I20260808 02:18:19.798107 1438 tablet_copy_service.cc:140] P aa78ed5a894742328b0e31dda1f3499a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 44c68eddae7e4a04b957b0ead4437ae8 ({username='oryx', principal='oryx/127.24.79.252@KRBTEST.COM'} at 127.0.0.1:41128)
I20260808 02:18:19.798243 1438 tablet_copy_service.cc:161] P aa78ed5a894742328b0e31dda1f3499a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 44c68eddae7e4a04b957b0ead4437ae8 at {username='oryx', principal='oryx/127.24.79.252@KRBTEST.COM'} at 127.0.0.1:41128: session id = 44c68eddae7e4a04b957b0ead4437ae8-00000000000000000000000000000000
I20260808 02:18:19.799451 1438 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 02:18:19.800999 2253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 02:18:19.803692 2253 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8: tablet copy: Starting download of 0 data blocks...
I20260808 02:18:19.803941 2253 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8: tablet copy: Starting download of 1 WAL segments...
I20260808 02:18:19.805830 2253 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 02:18:19.808039 1438 tablet_copy_service.cc:342] P aa78ed5a894742328b0e31dda1f3499a: Request end of tablet copy session 44c68eddae7e4a04b957b0ead4437ae8-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.79.252@KRBTEST.COM'} at 127.0.0.1:41128
I20260808 02:18:19.808154 1438 tablet_copy_service.cc:433] P aa78ed5a894742328b0e31dda1f3499a: ending tablet copy session 44c68eddae7e4a04b957b0ead4437ae8-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 44c68eddae7e4a04b957b0ead4437ae8
I20260808 02:18:19.817478 2253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:19.819384 2546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:19.819473 2547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:19.820571 2549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:19.820869 2253 server_base.cc:1034] running on GCE node
I20260808 02:18:19.821077 2253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:19.821427 2253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:19.822587 2253 hybrid_clock.cc:648] HybridClock initialized: now 1786155499822570 us; error 46 us; skew 500 ppm
Aug 08 02:18:19 dist-test-slave-1vmg krb5kdc[1339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155499, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:19.824434 2253 init.cc:377] Logged in from keytab as oryx/127.24.79.252@KRBTEST.COM (short username oryx)
I20260808 02:18:19.825016 2253 webserver.cc:460] Webserver started at http://127.24.79.252:38707/ using document root <none> and password file <none>
I20260808 02:18:19.825219 2253 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:19.825268 2253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:19.826814 2253 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 02:18:19.828016 2560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:19.828302 2253 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:18:19.828380 2253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/wal
uuid: "44c68eddae7e4a04b957b0ead4437ae8"
format_stamp: "Formatted at 2026-08-08 02:18:19 on dist-test-slave-1vmg"
server_key: "fe272037ebcc551dfd4910c6d8dddedd"
server_key_iv: "13730915abac62b5e188d96bb34025e5"
server_key_version: "encryptionkey@0"
I20260808 02:18:19.828464 2253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:19.852345 2253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:19.854550 2253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:19.859935 2253 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.252:44521
I20260808 02:18:19.859961 2641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.252:44521 every 8 connection(s)
I20260808 02:18:19.860613 2253 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/master-2/data/info.pb
I20260808 02:18:19.862589 2642 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:18:19.863853 2642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8: Bootstrap starting.
I20260808 02:18:19.873430 2642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:18:19.873780 2642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8: Bootstrap complete.
I20260808 02:18:19.874372 2642 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: NON_VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: true } }
I20260808 02:18:19.874508 2642 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 44c68eddae7e4a04b957b0ead4437ae8, State: Initialized, Role: LEARNER
I20260808 02:18:19.874609 2642 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: NON_VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: true } }
I20260808 02:18:19.875038 2657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: NON_VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: true } } }
I20260808 02:18:19.875141 2642 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:18:19.875273 2657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [sys.catalog]: This master's current role is: LEARNER
W20260808 02:18:19.875470 2518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:19.875844 2518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:19.875926 2518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 02:18:19.878760 2672 catalog_manager.cc:1259] Loaded cluster ID: 0a4525e343e5431e9aa0df4f93800b9c
I20260808 02:18:19.878821 2672 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8: loading cluster ID for follower catalog manager: success
I20260808 02:18:19.880494 2672 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8: acquiring CA information for follower catalog manager: success
W20260808 02:18:19.881165 2518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:19.881235 2518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:19.881311 2518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:18:19.881812 2672 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:18:19.887180 2518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.193.keytab
--principal=oryx/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:46067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=33151
--tserver_master_addrs=127.24.79.254:45167,127.24.79.253:38013,127.24.79.252:44521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:19.888393 2518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:19.889475 2518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:19.900065 2676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:19.900139 2518 server_base.cc:1034] running on GCE node
W20260808 02:18:19.900074 2677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:19.900065 2679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:19.900815 2518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:19.901500 2518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:19.902671 2518 hybrid_clock.cc:648] HybridClock initialized: now 1786155499902658 us; error 28 us; skew 500 ppm
Aug 08 02:18:19 dist-test-slave-1vmg krb5kdc[1339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155499, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:19.905568 2518 init.cc:377] Logged in from keytab as oryx/127.24.79.193@KRBTEST.COM (short username oryx)
I20260808 02:18:19.906554 2518 webserver.cc:460] Webserver started at http://127.24.79.193:33151/ using document root <none> and password file <none>
I20260808 02:18:19.907209 2518 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:19.907311 2518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:19.911500 2518 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:18:19.913698 2689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:19.915117 2518 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:19.915241 2518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "cbf75c267d9a4217bbf9c951246d6a93"
format_stamp: "Formatted at 2026-08-08 02:18:18 on dist-test-slave-1vmg"
server_key: "d254188a19d0de4dda986bf7c77cafb9"
server_key_iv: "8d4fbc2dcabfb854a8da6d4489016a3f"
server_key_version: "encryptionkey@0"
I20260808 02:18:19.915887 2518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:19.925251 2518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:19.928483 2518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:19.928670 2518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:19.929281 2518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:19.930238 2518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:19.930287 2518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:19.930330 2518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:19.930358 2518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:19.939486 2518 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:46067
I20260808 02:18:19.939515 2874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:46067 every 8 connection(s)
I20260808 02:18:19.940647 2518 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:18:19.947113 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 2518
I20260808 02:18:19.947319 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 1696
Aug 08 02:18:19 dist-test-slave-1vmg krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155499, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM
Aug 08 02:18:19 dist-test-slave-1vmg krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155499, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.253@KRBTEST.COM
I20260808 02:18:19.959127 1468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.942685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:57065 (local address 127.24.79.254:45167)
0808 02:18:19.942847 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:19.942853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:19.944328 (+ 1475us) server_negotiation.cc:407] Connection header received
0808 02:18:19.945262 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:19.945265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:19.945317 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:19.945391 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:19.946853 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.947732 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.949264 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.949536 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.949612 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.953844 (+ 4232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:19.953847 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:19.953850 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:19.953879 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:19.955830 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.956766 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.956770 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.956772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.956873 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.957604 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.957608 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.957610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.957879 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:19.957951 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:19.958776 (+ 825us) server_negotiation.cc:299] Negotiation successful
0808 02:18:19.958947 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 02:18:19.959640 2485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.944468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:60511 (local address 127.24.79.253:38013)
0808 02:18:19.944619 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:19.944625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:19.944714 (+ 89us) server_negotiation.cc:407] Connection header received
0808 02:18:19.945470 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:19.945474 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:19.945568 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:19.945666 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:19.946856 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.947702 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.952483 (+ 4781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.952848 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.952939 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.956132 (+ 3193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:19.956136 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:19.956139 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:19.956170 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:19.957770 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.958259 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.958263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.958264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.958357 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.958653 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.958656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.958659 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.958893 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:19.959009 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:19.959367 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 02:18:19.959498 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 02:18:19.961050 2885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.943158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.193:57065)
0808 02:18:19.944202 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:19.944219 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:19.944862 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:19.945518 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:19.945526 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:19.946102 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:19.946690 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.946698 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.947919 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.947922 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.949103 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.949114 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.949835 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.949841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.950123 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.951656 (+ 1533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:19.951680 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:19.953646 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:19.955975 (+ 2329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.955981 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.956002 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.956628 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.957243 (+ 615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.957246 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.957248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.957474 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.958037 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:19.958046 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:19.958471 (+ 425us) client_negotiation.cc:769] Sending connection context
0808 02:18:19.959430 (+ 959us) client_negotiation.cc:240] Negotiation successful
0808 02:18:19.959867 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":948,"mutex_wait_us":91,"spinlock_wait_cycles":53760,"thread_start_us":171,"threads_started":1}
I20260808 02:18:19.961050 2889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.944330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:38013 (local address 127.24.79.193:60511)
0808 02:18:19.944616 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:19.944630 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:19.944871 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:19.945695 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:19.945698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:19.946114 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:19.946688 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.946698 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.951468 (+ 4770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.951472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.952310 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.952323 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.953115 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.953119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.953217 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.954424 (+ 1207us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:19.954436 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:19.955957 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:19.957891 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.957894 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.957896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.958143 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.958470 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.958473 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.958475 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.958553 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.959082 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:19.959085 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:19.959179 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 02:18:19.959811 (+ 632us) client_negotiation.cc:240] Negotiation successful
0808 02:18:19.959933 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":49,"spinlock_wait_cycles":10880,"thread_start_us":92,"threads_started":1}
I20260808 02:18:19.962387 2876 heartbeater.cc:347] Connected to a master server at 127.24.79.254:45167
I20260808 02:18:19.962622 2876 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:19.963217 2876 heartbeater.cc:511] Master 127.24.79.254:45167 requested a full tablet report, sending...
I20260808 02:18:19.964596 1405 ts_manager.cc:194] Re-registered known tserver with Master: cbf75c267d9a4217bbf9c951246d6a93 (127.24.79.193:46067)
I20260808 02:18:19.965294 1405 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.79.193@KRBTEST.COM'} at 127.24.79.193:57065
I20260808 02:18:19.976372 2877 heartbeater.cc:347] Connected to a master server at 127.24.79.253:38013
I20260808 02:18:19.976639 2877 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:19.976997 2877 heartbeater.cc:511] Master 127.24.79.253:38013 requested a full tablet report, sending...
I20260808 02:18:19.978466 2290 ts_manager.cc:194] Registered new tserver with Master: cbf75c267d9a4217bbf9c951246d6a93 (127.24.79.193:46067)
I20260808 02:18:19.980125 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:41143
--local_ip_for_outbound_sockets=127.24.79.194
--tserver_master_addrs=127.24.79.254:45167
--webserver_port=46813
--webserver_interface=127.24.79.194
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.194.keytab
--principal=oryx/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.79.254:45167,127.24.79.253:38013
--tserver_master_addrs=127.24.79.254:45167,127.24.79.253:38013,127.24.79.252:44521 with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:18:19 dist-test-slave-1vmg krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155499, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.252@KRBTEST.COM
I20260808 02:18:19.984531 2879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.942654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:44683 (local address 127.24.79.252:44521)
0808 02:18:19.942981 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:19.942985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:19.943921 (+ 936us) server_negotiation.cc:407] Connection header received
0808 02:18:19.945121 (+ 1200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:19.945125 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:19.945194 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:19.945291 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:19.946899 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.948139 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.976735 (+ 28596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.977110 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.977218 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.981073 (+ 3855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:19.981078 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:19.981081 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:19.981117 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:19.982687 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.983308 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.983312 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.983313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.983400 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:19.983679 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:19.983681 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:19.983682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:19.983887 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:19.983961 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:19.984273 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 02:18:19.984391 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":90,"threads_started":1}
I20260808 02:18:19.984606 2880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:19.943158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:44521 (local address 127.24.79.193:44683)
0808 02:18:19.943747 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:19.943804 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:19.944860 (+ 1056us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:19.945519 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:19.945527 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:19.946080 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:19.946679 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.946694 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.975520 (+ 28826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.975525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.976530 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:19.976547 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:19.978509 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:19.978514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:19.978644 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:19.979333 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:19.979347 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:19.980878 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:19.982834 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.982839 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.982841 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.983196 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.983491 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:19.983494 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:19.983496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:19.983599 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:19.983977 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:19.983979 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:19.984116 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 02:18:19.984218 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 02:18:19.984404 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":20,"spinlock_wait_cycles":19840,"thread_start_us":595,"threads_started":1}
I20260808 02:18:19.985634 2875 heartbeater.cc:347] Connected to a master server at 127.24.79.252:44521
I20260808 02:18:19.985771 2875 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:19.985947 2875 heartbeater.cc:511] Master 127.24.79.252:44521 requested a full tablet report, sending...
I20260808 02:18:19.987481 2576 ts_manager.cc:194] Registered new tserver with Master: cbf75c267d9a4217bbf9c951246d6a93 (127.24.79.193:46067)
I20260808 02:18:19.989660 2084 heartbeater.cc:503] Master 127.24.79.254:45167 was elected leader, sending a full tablet report...
W20260808 02:18:20.105386 2890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:20.105813 2890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:20.105883 2890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:20.110144 2890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:20.110221 2890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:20.110283 2890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:18:20.116278 2890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.194.keytab
--principal=oryx/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:41143
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=46813
--tserver_master_addrs=127.24.79.254:45167,127.24.79.253:38013,127.24.79.252:44521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:20.117921 2890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:20.119458 2890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:20.132395 2901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:20.132416 2904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:20.132421 2902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:20.132982 2890 server_base.cc:1034] running on GCE node
I20260808 02:18:20.133495 2890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:20.134248 2890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:20.135423 2890 hybrid_clock.cc:648] HybridClock initialized: now 1786155500135387 us; error 56 us; skew 500 ppm
Aug 08 02:18:20 dist-test-slave-1vmg krb5kdc[1339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155500, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:20.138509 2890 init.cc:377] Logged in from keytab as oryx/127.24.79.194@KRBTEST.COM (short username oryx)
I20260808 02:18:20.139604 2890 webserver.cc:460] Webserver started at http://127.24.79.194:46813/ using document root <none> and password file <none>
I20260808 02:18:20.140240 2890 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:20.140327 2890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:20.144912 2890 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260808 02:18:20.147276 2917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:20.148999 2890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:18:20.149173 2890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "27a7405c7485416e8f2b14a5526d9178"
format_stamp: "Formatted at 2026-08-08 02:18:18 on dist-test-slave-1vmg"
server_key: "cf8090a1c1ac858fc3575a0b01d8ff2e"
server_key_iv: "7e9c0a7075546dffa172f020b7510a9e"
server_key_version: "encryptionkey@0"
I20260808 02:18:20.149775 2890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:20.162794 2890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:20.166373 2890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:20.166585 2890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:20.167325 2890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:20.168282 2890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:20.168332 2890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:20.168391 2890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:20.168421 2890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:20.178565 2890 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:41143
I20260808 02:18:20.178684 3109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:41143 every 8 connection(s)
I20260808 02:18:20.179814 2890 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:18:20.188988 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 2890
I20260808 02:18:20.189188 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 1900
Aug 08 02:18:20 dist-test-slave-1vmg krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155500, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.252@KRBTEST.COM
Aug 08 02:18:20 dist-test-slave-1vmg krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155500, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM
Aug 08 02:18:20 dist-test-slave-1vmg krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155500, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.253@KRBTEST.COM
I20260808 02:18:20.197890 2879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.181872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:50983 (local address 127.24.79.252:44521)
0808 02:18:20.182012 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:20.182017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:20.183139 (+ 1122us) server_negotiation.cc:407] Connection header received
0808 02:18:20.184602 (+ 1463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:20.184605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:20.184655 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:20.184720 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:20.186144 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.186706 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.188474 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.188875 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.189121 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.193350 (+ 4229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:20.193355 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:20.193358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:20.193392 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:20.195129 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.195858 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.195862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.195864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.195962 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.196400 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.196404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.196405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.196668 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:20.196751 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:20.197556 (+ 805us) server_negotiation.cc:299] Negotiation successful
0808 02:18:20.197723 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 02:18:20.197953 2485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.183740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:51485 (local address 127.24.79.253:38013)
0808 02:18:20.183878 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:20.183883 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:20.183923 (+ 40us) server_negotiation.cc:407] Connection header received
0808 02:18:20.184852 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:20.184855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:20.184910 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:20.184991 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:20.187118 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.187814 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.188845 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.189156 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.189244 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.194254 (+ 5010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:20.194259 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:20.194262 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:20.194294 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:20.195939 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.196505 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.196509 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.196510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.196615 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.196956 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.196960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.196962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.197226 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:20.197303 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:20.197662 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 02:18:20.197791 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 02:18:20.199364 1468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.181869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:45259 (local address 127.24.79.254:45167)
0808 02:18:20.182012 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:20.182016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:20.183258 (+ 1242us) server_negotiation.cc:407] Connection header received
0808 02:18:20.184343 (+ 1085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:20.184347 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:20.184407 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:20.184474 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:20.186874 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.187705 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.188797 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.189104 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.189178 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.195724 (+ 6546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:20.195728 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:20.195731 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:20.195765 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:20.197416 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.197916 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.197919 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.197921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.198013 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.198408 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.198412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.198414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.198649 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:20.198722 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:20.199100 (+ 378us) server_negotiation.cc:299] Negotiation successful
0808 02:18:20.199227 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 02:18:20.199864 3117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.182361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.194:45259)
0808 02:18:20.183146 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:20.183161 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:20.184128 (+ 967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:20.184669 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:20.184677 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:20.185206 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:20.185951 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:20.185972 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.187855 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.187859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:20.188650 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:20.188660 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.189339 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.189342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:20.189577 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.190394 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:20.190407 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:20.195538 (+ 5131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:20.197536 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:20.197539 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:20.197540 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:20.197801 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:20.198205 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:20.198208 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:20.198210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:20.198301 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:20.198795 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:20.198798 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:20.198887 (+ 89us) client_negotiation.cc:769] Sending connection context
0808 02:18:20.199514 (+ 627us) client_negotiation.cc:240] Negotiation successful
0808 02:18:20.199667 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"mutex_wait_us":205,"spinlock_wait_cycles":55936,"thread_start_us":133,"threads_started":1}
I20260808 02:18:20.199880 3120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.183529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:38013 (local address 127.24.79.194:51485)
0808 02:18:20.183857 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:20.183868 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:20.184121 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:20.185031 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:20.185034 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:20.185244 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:20.185950 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:20.185972 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.187937 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.187941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:20.188698 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:20.188709 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.191830 (+ 3121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.191834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:20.191942 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.192486 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:20.192502 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:20.194079 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:20.196085 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:20.196088 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:20.196090 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:20.196377 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:20.196742 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:20.196745 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:20.196746 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:20.196841 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:20.197379 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:20.197382 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:20.197474 (+ 92us) client_negotiation.cc:769] Sending connection context
0808 02:18:20.198190 (+ 716us) client_negotiation.cc:240] Negotiation successful
0808 02:18:20.198616 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":249,"spinlock_wait_cycles":38016,"thread_start_us":115,"threads_started":1}
I20260808 02:18:20.200172 3119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.182361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:44521 (local address 127.24.79.194:50983)
0808 02:18:20.182960 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:20.183015 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:20.184116 (+ 1101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:20.184765 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:20.184768 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:20.185195 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:20.185949 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:20.185970 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.186870 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.186873 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:20.188329 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:20.188341 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.189339 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.189342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:20.189756 (+ 414us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.190356 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:20.190382 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:20.193100 (+ 2718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:20.195275 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:20.195282 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:20.195306 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:20.195728 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:20.196068 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:20.196070 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:20.196072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:20.196284 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:20.196825 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:20.196832 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:20.197251 (+ 419us) client_negotiation.cc:769] Sending connection context
0808 02:18:20.198065 (+ 814us) client_negotiation.cc:240] Negotiation successful
0808 02:18:20.198529 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":500,"spinlock_wait_cycles":16512,"thread_start_us":133,"threads_started":1}
I20260808 02:18:20.201181 3112 heartbeater.cc:347] Connected to a master server at 127.24.79.254:45167
I20260808 02:18:20.201197 3111 heartbeater.cc:347] Connected to a master server at 127.24.79.252:44521
I20260808 02:18:20.201251 3113 heartbeater.cc:347] Connected to a master server at 127.24.79.253:38013
I20260808 02:18:20.201459 3111 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:20.201459 3113 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:20.201459 3112 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:20.202133 3111 heartbeater.cc:511] Master 127.24.79.252:44521 requested a full tablet report, sending...
I20260808 02:18:20.202133 3112 heartbeater.cc:511] Master 127.24.79.254:45167 requested a full tablet report, sending...
I20260808 02:18:20.202133 3113 heartbeater.cc:511] Master 127.24.79.253:38013 requested a full tablet report, sending...
I20260808 02:18:20.203402 1405 ts_manager.cc:194] Re-registered known tserver with Master: 27a7405c7485416e8f2b14a5526d9178 (127.24.79.194:41143)
I20260808 02:18:20.203472 2576 ts_manager.cc:194] Registered new tserver with Master: 27a7405c7485416e8f2b14a5526d9178 (127.24.79.194:41143)
I20260808 02:18:20.204128 1405 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.79.194@KRBTEST.COM'} at 127.24.79.194:45259
I20260808 02:18:20.204145 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:35579
--local_ip_for_outbound_sockets=127.24.79.195
--tserver_master_addrs=127.24.79.254:45167
--webserver_port=34149
--webserver_interface=127.24.79.195
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.195.keytab
--principal=oryx/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.79.254:45167,127.24.79.253:38013
--tserver_master_addrs=127.24.79.254:45167,127.24.79.253:38013,127.24.79.252:44521 with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:18:20.204654 2290 ts_manager.cc:194] Registered new tserver with Master: 27a7405c7485416e8f2b14a5526d9178 (127.24.79.194:41143)
I20260808 02:18:20.293311 2879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.289025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39836 (local address 127.24.79.252:44521)
0808 02:18:20.289191 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:20.289196 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:20.289287 (+ 91us) server_negotiation.cc:407] Connection header received
0808 02:18:20.289429 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:20.289434 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:20.289517 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:20.289667 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:20.289942 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.290769 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.291963 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.292716 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.292841 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.293037 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:20.293083 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 02:18:20.293164 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 02:18:20.293315 3123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.288903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:44521 (local address 127.0.0.1:39836)
0808 02:18:20.289180 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:20.289195 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:20.289326 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:20.289690 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:20.289692 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:20.289713 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:20.289830 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:20.289835 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.290962 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.290967 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:20.291804 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:20.291818 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.292876 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.292879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:20.292968 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.292997 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 02:18:20.293060 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 02:18:20.293136 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":87,"threads_started":1}
I20260808 02:18:20.294238 2601 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 02:18:20.294682 2657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa78ed5a894742328b0e31dda1f3499a. Latest consensus state: current_term: 1 leader_uuid: "aa78ed5a894742328b0e31dda1f3499a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: NON_VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: true } } }
I20260808 02:18:20.294839 2657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [sys.catalog]: This master's current role is: LEARNER
W20260808 02:18:20.328272 3121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:20.328547 3121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:20.328613 3121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:20.332932 3121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:20.333024 3121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:20.333117 3121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:18:20.338847 3121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.195.keytab
--principal=oryx/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:35579
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=34149
--tserver_master_addrs=127.24.79.254:45167,127.24.79.253:38013,127.24.79.252:44521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:20.340168 3121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:20.341305 3121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:20.351919 3133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:20.351971 3121 server_base.cc:1034] running on GCE node
W20260808 02:18:20.351907 3130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:20.351910 3135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:20.352579 3121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:20.353343 3121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:20.354512 3121 hybrid_clock.cc:648] HybridClock initialized: now 1786155500354468 us; error 60 us; skew 500 ppm
Aug 08 02:18:20 dist-test-slave-1vmg krb5kdc[1339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155500, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:20.357721 3121 init.cc:377] Logged in from keytab as oryx/127.24.79.195@KRBTEST.COM (short username oryx)
I20260808 02:18:20.358747 3121 webserver.cc:460] Webserver started at http://127.24.79.195:34149/ using document root <none> and password file <none>
I20260808 02:18:20.359409 3121 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:20.359510 3121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:20.363852 3121 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:18:20.366130 3146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:20.367614 3121 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:20.367762 3121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "053e2be59b2d4d06ae90fc3bff12aebb"
format_stamp: "Formatted at 2026-08-08 02:18:18 on dist-test-slave-1vmg"
server_key: "2a1e6388e182530431a07643b14e0028"
server_key_iv: "e4bf038160bb31e741015e157bb694b7"
server_key_version: "encryptionkey@0"
I20260808 02:18:20.368423 3121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:20.391561 3121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:20.395536 3121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:20.395759 3121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:20.396454 3121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:20.397576 3121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:20.397634 3121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:20.397694 3121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:20.397722 3121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:20.407502 3121 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:35579
I20260808 02:18:20.407521 3312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:35579 every 8 connection(s)
I20260808 02:18:20.408703 3121 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:18:20.413312 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 3121
Aug 08 02:18:20 dist-test-slave-1vmg krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155500, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.252@KRBTEST.COM
Aug 08 02:18:20 dist-test-slave-1vmg krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155500, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM
Aug 08 02:18:20 dist-test-slave-1vmg krb5kdc[1339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155500, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.253@KRBTEST.COM
I20260808 02:18:20.430370 1468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.410792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:37983 (local address 127.24.79.254:45167)
0808 02:18:20.410961 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:20.410965 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:20.412266 (+ 1301us) server_negotiation.cc:407] Connection header received
0808 02:18:20.413349 (+ 1083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:20.413354 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:20.413417 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:20.413473 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:20.415787 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.416530 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.418782 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.419130 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.419208 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.424720 (+ 5512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:20.424724 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:20.424727 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:20.424758 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:20.426984 (+ 2226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.427707 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.427711 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.427712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.427804 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.428878 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.428882 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.428884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.429144 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:20.429226 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:20.430056 (+ 830us) server_negotiation.cc:299] Negotiation successful
0808 02:18:20.430200 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 02:18:20.430610 3333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.415663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41140 (local address 127.24.79.254:45167)
0808 02:18:20.417502 (+ 1839us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:20.417506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:20.417519 (+ 13us) server_negotiation.cc:407] Connection header received
0808 02:18:20.417560 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:20.417563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:20.417613 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:20.417695 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:20.418822 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.419565 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.420698 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.421062 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.421163 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.423554 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:20.423557 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:20.423560 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:20.423590 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:20.428459 (+ 4869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.428924 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.428927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.428929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.429000 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.429286 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.429289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.429291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.429488 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:20.429607 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:20.430349 (+ 742us) server_negotiation.cc:299] Negotiation successful
0808 02:18:20.430469 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1756,"thread_start_us":89,"threads_started":1}
I20260808 02:18:20.431418 2879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.410790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:43385 (local address 127.24.79.252:44521)
0808 02:18:20.410958 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:20.410963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:20.412529 (+ 1566us) server_negotiation.cc:407] Connection header received
0808 02:18:20.413376 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:20.413380 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:20.413438 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:20.413487 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:20.415589 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.416478 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.418604 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.418954 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.419052 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.424641 (+ 5589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:20.424645 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:20.424647 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:20.424675 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:20.428437 (+ 3762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.429240 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.429244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.429246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.429350 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.430387 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.430391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.430392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.430607 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:20.430693 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:20.431143 (+ 450us) server_negotiation.cc:299] Negotiation successful
0808 02:18:20.431276 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 02:18:20.432389 3326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.412761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:38013 (local address 127.24.79.195:49367)
0808 02:18:20.413021 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:20.413036 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:20.413144 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:20.414836 (+ 1692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:20.414840 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:20.415040 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:20.415411 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:20.415432 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.416719 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.416722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:20.418471 (+ 1749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:20.418485 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.421311 (+ 2826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.421315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:20.421403 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.422000 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:20.422016 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:20.424508 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:20.428132 (+ 3624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:20.428135 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:20.428137 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:20.429120 (+ 983us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:20.429905 (+ 785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:20.429909 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:20.429911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:20.429998 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:20.430463 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:20.430466 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:20.430561 (+ 95us) client_negotiation.cc:769] Sending connection context
0808 02:18:20.430680 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 02:18:20.431152 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":258,"thread_start_us":77,"threads_started":1}
I20260808 02:18:20.433058 3337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.416678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39840 (local address 127.24.79.252:44521)
0808 02:18:20.418439 (+ 1761us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:20.418443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:20.418842 (+ 399us) server_negotiation.cc:407] Connection header received
0808 02:18:20.418886 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:20.418889 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:20.418974 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:20.419108 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:20.420635 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.421403 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.422551 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.423690 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.423875 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.426034 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:20.426038 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:20.426041 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:20.426070 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:20.427801 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.428634 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.428638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.428640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.428725 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.431256 (+ 2531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.431260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.431261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.432064 (+ 803us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:20.432203 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:20.432765 (+ 562us) server_negotiation.cc:299] Negotiation successful
0808 02:18:20.432908 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1675,"thread_start_us":74,"threads_started":1}
I20260808 02:18:20.433310 3338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.416933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52416 (local address 127.24.79.253:38013)
0808 02:18:20.420110 (+ 3177us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:20.420158 (+ 48us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:20.421878 (+ 1720us) server_negotiation.cc:407] Connection header received
0808 02:18:20.422075 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:20.422078 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:20.422134 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:20.422200 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:20.423148 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.423895 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.425594 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.425936 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.426013 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.427779 (+ 1766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:20.427784 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:20.427787 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:20.427817 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:20.429612 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.430104 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.430107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.430109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.430268 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.431846 (+ 1578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.431850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.431851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.432061 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:20.432466 (+ 405us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:20.433008 (+ 542us) server_negotiation.cc:299] Negotiation successful
0808 02:18:20.433137 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3091,"spinlock_wait_cycles":8448,"thread_start_us":72,"threads_started":1}
I20260808 02:18:20.433594 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.411383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:44521 (local address 127.24.79.195:43385)
0808 02:18:20.412423 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:20.412436 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:20.413071 (+ 635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:20.413848 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:20.413858 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:20.414540 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:20.415409 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:20.415430 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.416852 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.416855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:20.418472 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:20.418485 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.419292 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.419296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:20.419512 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.420411 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:20.420433 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:20.424427 (+ 3994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:20.428564 (+ 4137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:20.428568 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:20.428570 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:20.428839 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:20.429488 (+ 649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:20.429491 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:20.429494 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:20.429586 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:20.430737 (+ 1151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:20.430740 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:20.430825 (+ 85us) client_negotiation.cc:769] Sending connection context
0808 02:18:20.430909 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 02:18:20.433089 (+ 2180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":945,"mutex_wait_us":2669,"spinlock_wait_cycles":60288,"thread_start_us":221,"threads_started":1}
I20260808 02:18:20.434274 2485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.414323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:49367 (local address 127.24.79.253:38013)
0808 02:18:20.414468 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:20.414473 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:20.414489 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:18:20.414536 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:20.414539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:20.414583 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:20.414670 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:20.415624 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.416462 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.419997 (+ 3535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.420356 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.420471 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.424641 (+ 4170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:20.424645 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:20.424647 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:20.424675 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:20.428007 (+ 3332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.429692 (+ 1685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.429695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.429697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.429798 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.430104 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.430107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.430109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.430339 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:20.430417 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:20.434025 (+ 3608us) server_negotiation.cc:299] Negotiation successful
0808 02:18:20.434142 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 02:18:20.434981 3324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.411383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.195:37983)
0808 02:18:20.412061 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:20.412109 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:20.413063 (+ 954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:20.413848 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:20.413858 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:20.414542 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:20.415645 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:20.415653 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.416726 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.416729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:20.418651 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:20.418659 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.419371 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.419375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:20.419514 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.420645 (+ 1131us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:20.420660 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:20.424549 (+ 3889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:20.427130 (+ 2581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:20.427137 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:20.427160 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:20.427557 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:20.427918 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:20.427921 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:20.427923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:20.428534 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:20.429332 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:20.429340 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:20.429733 (+ 393us) client_negotiation.cc:769] Sending connection context
0808 02:18:20.430886 (+ 1153us) client_negotiation.cc:240] Negotiation successful
0808 02:18:20.434768 (+ 3882us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":4565,"spinlock_wait_cycles":2816,"thread_start_us":692,"threads_started":1}
I20260808 02:18:20.435461 3319 heartbeater.cc:347] Connected to a master server at 127.24.79.253:38013
I20260808 02:18:20.435492 3316 heartbeater.cc:347] Connected to a master server at 127.24.79.252:44521
I20260808 02:18:20.435750 3319 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:20.435750 3316 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:20.436044 3317 heartbeater.cc:347] Connected to a master server at 127.24.79.254:45167
I20260808 02:18:20.436118 3317 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:20.436460 3317 heartbeater.cc:511] Master 127.24.79.254:45167 requested a full tablet report, sending...
I20260808 02:18:20.436460 3316 heartbeater.cc:511] Master 127.24.79.252:44521 requested a full tablet report, sending...
I20260808 02:18:20.436525 3319 heartbeater.cc:511] Master 127.24.79.253:38013 requested a full tablet report, sending...
I20260808 02:18:20.433981 1405 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41140:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 02:18:20.437714 1411 ts_manager.cc:194] Re-registered known tserver with Master: 053e2be59b2d4d06ae90fc3bff12aebb (127.24.79.195:35579)
I20260808 02:18:20.437789 2294 ts_manager.cc:194] Registered new tserver with Master: 053e2be59b2d4d06ae90fc3bff12aebb (127.24.79.195:35579)
I20260808 02:18:20.437789 2576 ts_manager.cc:194] Registered new tserver with Master: 053e2be59b2d4d06ae90fc3bff12aebb (127.24.79.195:35579)
W20260808 02:18:20.438076 1405 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:18:20.438483 1411 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.79.195@KRBTEST.COM'} at 127.24.79.195:37983
I20260808 02:18:20.441066 2248 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aa78ed5a894742328b0e31dda1f3499a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: NON_VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: true } } }
I20260808 02:18:20.441236 2248 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:20.443609 2248 raft_consensus.cc:1050] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a: attempting to promote NON_VOTER 44c68eddae7e4a04b957b0ead4437ae8 to VOTER
I20260808 02:18:20.443990 2248 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: false } }
I20260808 02:18:20.445248 2601 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 1 LEARNER]: Refusing update from remote peer aa78ed5a894742328b0e31dda1f3499a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 02:18:20.445258 2311 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 1 FOLLOWER]: Refusing update from remote peer aa78ed5a894742328b0e31dda1f3499a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 02:18:20.445660 3345 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [LEADER]: Connected to new peer: Peer: permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 02:18:20.445827 2248 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 02:18:20.447772 2248 raft_consensus.cc:2932] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 44c68eddae7e4a04b957b0ead4437ae8 (127.24.79.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: false } } }
I20260808 02:18:20.448246 2311 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 44c68eddae7e4a04b957b0ead4437ae8 (127.24.79.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: false } } }
I20260808 02:18:20.448407 2601 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 44c68eddae7e4a04b957b0ead4437ae8 (127.24.79.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: false } } }
I20260808 02:18:20.449751 3348 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "aa78ed5a894742328b0e31dda1f3499a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: false } } }
I20260808 02:18:20.449880 3348 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:20.450105 3348 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aa78ed5a894742328b0e31dda1f3499a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: false } } }
I20260808 02:18:20.450201 3348 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:20.450233 2657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aa78ed5a894742328b0e31dda1f3499a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: false } } }
I20260808 02:18:20.450336 2657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:18:20.450088 3358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aa78ed5a894742328b0e31dda1f3499a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: false } } }
I20260808 02:18:20.450470 2248 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aa78ed5a894742328b0e31dda1f3499a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: false } } }
I20260808 02:18:20.450527 3358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:18:20.450582 2248 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:20.467013 3369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.462559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48244 (local address 127.24.79.193:46067)
0808 02:18:20.462857 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:20.462863 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:20.462887 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:18:20.462988 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:20.462993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:20.463179 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:20.463333 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:20.463742 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.464683 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.465727 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.466324 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.466430 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.466613 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:20.466739 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 02:18:20.466859 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":85,"threads_started":1}
I20260808 02:18:20.467007 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.462116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:46067 (local address 127.0.0.1:48244)
0808 02:18:20.462357 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:20.462369 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:20.462468 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:20.463482 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:20.463484 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:20.463498 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:20.463603 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:20.463609 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.464803 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.464807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:20.465598 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:20.465608 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.466454 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.466458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:20.466604 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.466631 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 02:18:20.466705 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 02:18:20.466791 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":29,"spinlock_wait_cycles":1280,"thread_start_us":105,"threads_started":1}
I20260808 02:18:20.468170 3361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.461675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:35579 (local address 127.0.0.1:37290)
0808 02:18:20.462260 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:20.462273 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:20.462467 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:20.463140 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:20.463143 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:20.463162 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:20.463306 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:20.463314 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.465269 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.465274 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:20.466574 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:20.466586 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.467758 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.467761 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:20.467854 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.467882 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 02:18:20.467935 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 02:18:20.468010 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":1,"spinlock_wait_cycles":2816,"thread_start_us":154,"threads_started":1}
I20260808 02:18:20.468977 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.461861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37290 (local address 127.24.79.195:35579)
0808 02:18:20.462187 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:20.462201 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:20.462375 (+ 174us) server_negotiation.cc:407] Connection header received
0808 02:18:20.462669 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:20.462676 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:20.462981 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:20.463175 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:20.463414 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.465132 (+ 1718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.466745 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.467608 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.468481 (+ 873us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.468667 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:20.468725 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 02:18:20.468831 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":121,"threads_started":1}
I20260808 02:18:20.470887 2765 tablet_service.cc:1467] Processing CreateTablet for tablet a1ab98364a494a76bc429bb90fe513fc (DEFAULT_TABLE table=test-table [id=1178745ff0dd45a3a7a34d1653a13d0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:20.472390 2765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1ab98364a494a76bc429bb90fe513fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:20.472167 3227 tablet_service.cc:1467] Processing CreateTablet for tablet a1ab98364a494a76bc429bb90fe513fc (DEFAULT_TABLE table=test-table [id=1178745ff0dd45a3a7a34d1653a13d0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:20.472931 3123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.461435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:41143 (local address 127.0.0.1:49278)
0808 02:18:20.461623 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:20.461637 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:20.461715 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:20.468519 (+ 6804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:20.468522 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:20.468545 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:20.468651 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:20.468662 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.470649 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.470653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:20.471479 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:20.471492 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.472458 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.472462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:20.472557 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.472583 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 02:18:20.472665 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 02:18:20.472765 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260808 02:18:20.472987 3360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.461607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49278 (local address 127.24.79.194:41143)
0808 02:18:20.467903 (+ 6296us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:20.467922 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:20.467950 (+ 28us) server_negotiation.cc:407] Connection header received
0808 02:18:20.468039 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:20.468044 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:20.468307 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:20.468835 (+ 528us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:20.468997 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.470519 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.471633 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.472325 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.472432 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.472597 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:20.472748 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 02:18:20.472853 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6166,"thread_start_us":169,"threads_started":1}
I20260808 02:18:20.474684 3227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1ab98364a494a76bc429bb90fe513fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:20.476080 3004 tablet_service.cc:1467] Processing CreateTablet for tablet a1ab98364a494a76bc429bb90fe513fc (DEFAULT_TABLE table=test-table [id=1178745ff0dd45a3a7a34d1653a13d0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:20.477159 3004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1ab98364a494a76bc429bb90fe513fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:20.482478 3370 tablet_bootstrap.cc:492] T a1ab98364a494a76bc429bb90fe513fc P cbf75c267d9a4217bbf9c951246d6a93: Bootstrap starting.
I20260808 02:18:20.485239 3372 tablet_bootstrap.cc:492] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178: Bootstrap starting.
I20260808 02:18:20.485713 3370 tablet_bootstrap.cc:654] T a1ab98364a494a76bc429bb90fe513fc P cbf75c267d9a4217bbf9c951246d6a93: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:20.486653 3374 tablet_bootstrap.cc:492] T a1ab98364a494a76bc429bb90fe513fc P 053e2be59b2d4d06ae90fc3bff12aebb: Bootstrap starting.
I20260808 02:18:20.486936 3370 log.cc:824] T a1ab98364a494a76bc429bb90fe513fc P cbf75c267d9a4217bbf9c951246d6a93: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:20.487856 3372 tablet_bootstrap.cc:654] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:20.488643 3372 log.cc:824] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:20.489326 3374 tablet_bootstrap.cc:654] T a1ab98364a494a76bc429bb90fe513fc P 053e2be59b2d4d06ae90fc3bff12aebb: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:20.489568 3370 tablet_bootstrap.cc:492] T a1ab98364a494a76bc429bb90fe513fc P cbf75c267d9a4217bbf9c951246d6a93: No bootstrap required, opened a new log
I20260808 02:18:20.489823 3370 ts_tablet_manager.cc:1397] T a1ab98364a494a76bc429bb90fe513fc P cbf75c267d9a4217bbf9c951246d6a93: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260808 02:18:20.490223 3374 log.cc:824] T a1ab98364a494a76bc429bb90fe513fc P 053e2be59b2d4d06ae90fc3bff12aebb: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:20.490899 3372 tablet_bootstrap.cc:492] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178: No bootstrap required, opened a new log
I20260808 02:18:20.491169 3372 ts_tablet_manager.cc:1397] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.002s
I20260808 02:18:20.492940 3374 tablet_bootstrap.cc:492] T a1ab98364a494a76bc429bb90fe513fc P 053e2be59b2d4d06ae90fc3bff12aebb: No bootstrap required, opened a new log
I20260808 02:18:20.493220 3374 ts_tablet_manager.cc:1397] T a1ab98364a494a76bc429bb90fe513fc P 053e2be59b2d4d06ae90fc3bff12aebb: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 02:18:20.493984 3370 raft_consensus.cc:348] T a1ab98364a494a76bc429bb90fe513fc P cbf75c267d9a4217bbf9c951246d6a93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e2be59b2d4d06ae90fc3bff12aebb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 35579 } } peers { permanent_uuid: "27a7405c7485416e8f2b14a5526d9178" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41143 } } peers { permanent_uuid: "cbf75c267d9a4217bbf9c951246d6a93" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46067 } }
I20260808 02:18:20.494333 3370 raft_consensus.cc:374] T a1ab98364a494a76bc429bb90fe513fc P cbf75c267d9a4217bbf9c951246d6a93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:20.494401 3370 raft_consensus.cc:729] T a1ab98364a494a76bc429bb90fe513fc P cbf75c267d9a4217bbf9c951246d6a93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbf75c267d9a4217bbf9c951246d6a93, State: Initialized, Role: FOLLOWER
I20260808 02:18:20.494988 3370 consensus_queue.cc:260] T a1ab98364a494a76bc429bb90fe513fc P cbf75c267d9a4217bbf9c951246d6a93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e2be59b2d4d06ae90fc3bff12aebb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 35579 } } peers { permanent_uuid: "27a7405c7485416e8f2b14a5526d9178" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41143 } } peers { permanent_uuid: "cbf75c267d9a4217bbf9c951246d6a93" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46067 } }
I20260808 02:18:20.495124 3372 raft_consensus.cc:348] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e2be59b2d4d06ae90fc3bff12aebb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 35579 } } peers { permanent_uuid: "27a7405c7485416e8f2b14a5526d9178" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41143 } } peers { permanent_uuid: "cbf75c267d9a4217bbf9c951246d6a93" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46067 } }
I20260808 02:18:20.495419 3372 raft_consensus.cc:374] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:20.495486 3372 raft_consensus.cc:729] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27a7405c7485416e8f2b14a5526d9178, State: Initialized, Role: FOLLOWER
I20260808 02:18:20.495999 3372 consensus_queue.cc:260] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e2be59b2d4d06ae90fc3bff12aebb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 35579 } } peers { permanent_uuid: "27a7405c7485416e8f2b14a5526d9178" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41143 } } peers { permanent_uuid: "cbf75c267d9a4217bbf9c951246d6a93" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46067 } }
I20260808 02:18:20.496192 3370 ts_tablet_manager.cc:1428] T a1ab98364a494a76bc429bb90fe513fc P cbf75c267d9a4217bbf9c951246d6a93: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 02:18:20.496562 2876 heartbeater.cc:503] Master 127.24.79.254:45167 was elected leader, sending a full tablet report...
I20260808 02:18:20.497604 3372 ts_tablet_manager.cc:1428] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260808 02:18:20.497468 3374 raft_consensus.cc:348] T a1ab98364a494a76bc429bb90fe513fc P 053e2be59b2d4d06ae90fc3bff12aebb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e2be59b2d4d06ae90fc3bff12aebb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 35579 } } peers { permanent_uuid: "27a7405c7485416e8f2b14a5526d9178" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41143 } } peers { permanent_uuid: "cbf75c267d9a4217bbf9c951246d6a93" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46067 } }
I20260808 02:18:20.497774 3374 raft_consensus.cc:374] T a1ab98364a494a76bc429bb90fe513fc P 053e2be59b2d4d06ae90fc3bff12aebb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:20.497835 3374 raft_consensus.cc:729] T a1ab98364a494a76bc429bb90fe513fc P 053e2be59b2d4d06ae90fc3bff12aebb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 053e2be59b2d4d06ae90fc3bff12aebb, State: Initialized, Role: FOLLOWER
I20260808 02:18:20.499064 3112 heartbeater.cc:503] Master 127.24.79.254:45167 was elected leader, sending a full tablet report...
I20260808 02:18:20.500469 3374 consensus_queue.cc:260] T a1ab98364a494a76bc429bb90fe513fc P 053e2be59b2d4d06ae90fc3bff12aebb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e2be59b2d4d06ae90fc3bff12aebb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 35579 } } peers { permanent_uuid: "27a7405c7485416e8f2b14a5526d9178" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41143 } } peers { permanent_uuid: "cbf75c267d9a4217bbf9c951246d6a93" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46067 } }
I20260808 02:18:20.502795 3317 heartbeater.cc:503] Master 127.24.79.254:45167 was elected leader, sending a full tablet report...
I20260808 02:18:20.503717 3374 ts_tablet_manager.cc:1428] T a1ab98364a494a76bc429bb90fe513fc P 053e2be59b2d4d06ae90fc3bff12aebb: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260808 02:18:20.632659 3380 raft_consensus.cc:482] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:18:20.632920 3380 raft_consensus.cc:504] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e2be59b2d4d06ae90fc3bff12aebb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 35579 } } peers { permanent_uuid: "27a7405c7485416e8f2b14a5526d9178" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41143 } } peers { permanent_uuid: "cbf75c267d9a4217bbf9c951246d6a93" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46067 } }
I20260808 02:18:20.634159 3380 leader_election.cc:288] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 053e2be59b2d4d06ae90fc3bff12aebb (127.24.79.195:35579), cbf75c267d9a4217bbf9c951246d6a93 (127.24.79.193:46067)
I20260808 02:18:20.638159 3369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.634620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:42651 (local address 127.24.79.193:46067)
0808 02:18:20.634786 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:20.634791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:20.634808 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:18:20.634868 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:20.634871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:20.634948 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:20.635043 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:20.635400 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.636037 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.637200 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.637607 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.637722 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.637781 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:20.637950 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 02:18:20.638026 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 02:18:20.638202 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.634714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:36697 (local address 127.24.79.195:35579)
0808 02:18:20.634868 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:20.634873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:20.635149 (+ 276us) server_negotiation.cc:407] Connection header received
0808 02:18:20.635206 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:20.635208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:20.635257 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:20.635320 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:20.635585 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.636098 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.637269 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.637672 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.637784 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.637838 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:20.637999 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 02:18:20.638078 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 02:18:20.638212 3117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.634510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:46067 (local address 127.24.79.194:42651)
0808 02:18:20.634632 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:20.634647 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:20.634745 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:20.635073 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:20.635076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:20.635099 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:20.635250 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:20.635257 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.636168 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.636172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:20.637063 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:20.637074 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.637735 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.637739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:20.637835 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.637857 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 02:18:20.637922 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 02:18:20.637999 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":2}
I20260808 02:18:20.638223 3119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.634510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:35579 (local address 127.24.79.194:36697)
0808 02:18:20.634976 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:20.634990 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:20.635069 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:20.635355 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:20.635358 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:20.635374 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:20.635481 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:20.635486 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.636212 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.636215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:20.637138 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:20.637149 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.637789 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.637792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:20.637890 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.637907 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 02:18:20.637972 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 02:18:20.638044 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":301}
I20260808 02:18:20.639019 2803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1ab98364a494a76bc429bb90fe513fc" candidate_uuid: "27a7405c7485416e8f2b14a5526d9178" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbf75c267d9a4217bbf9c951246d6a93" is_pre_election: true
I20260808 02:18:20.639016 3254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1ab98364a494a76bc429bb90fe513fc" candidate_uuid: "27a7405c7485416e8f2b14a5526d9178" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "053e2be59b2d4d06ae90fc3bff12aebb" is_pre_election: true
I20260808 02:18:20.639349 3254 raft_consensus.cc:2454] T a1ab98364a494a76bc429bb90fe513fc P 053e2be59b2d4d06ae90fc3bff12aebb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27a7405c7485416e8f2b14a5526d9178 in term 0.
I20260808 02:18:20.639348 2803 raft_consensus.cc:2454] T a1ab98364a494a76bc429bb90fe513fc P cbf75c267d9a4217bbf9c951246d6a93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27a7405c7485416e8f2b14a5526d9178 in term 0.
I20260808 02:18:20.639840 2921 leader_election.cc:302] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 053e2be59b2d4d06ae90fc3bff12aebb, 27a7405c7485416e8f2b14a5526d9178; no voters:
I20260808 02:18:20.640128 3380 raft_consensus.cc:2781] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:18:20.640220 3380 raft_consensus.cc:482] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:18:20.640280 3380 raft_consensus.cc:3037] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:20.641397 3380 raft_consensus.cc:504] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e2be59b2d4d06ae90fc3bff12aebb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 35579 } } peers { permanent_uuid: "27a7405c7485416e8f2b14a5526d9178" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41143 } } peers { permanent_uuid: "cbf75c267d9a4217bbf9c951246d6a93" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46067 } }
I20260808 02:18:20.641759 3380 leader_election.cc:288] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178 [CANDIDATE]: Term 1 election: Requested vote from peers 053e2be59b2d4d06ae90fc3bff12aebb (127.24.79.195:35579), cbf75c267d9a4217bbf9c951246d6a93 (127.24.79.193:46067)
I20260808 02:18:20.642165 3254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1ab98364a494a76bc429bb90fe513fc" candidate_uuid: "27a7405c7485416e8f2b14a5526d9178" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "053e2be59b2d4d06ae90fc3bff12aebb"
I20260808 02:18:20.642242 2803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1ab98364a494a76bc429bb90fe513fc" candidate_uuid: "27a7405c7485416e8f2b14a5526d9178" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbf75c267d9a4217bbf9c951246d6a93"
I20260808 02:18:20.642303 3254 raft_consensus.cc:3037] T a1ab98364a494a76bc429bb90fe513fc P 053e2be59b2d4d06ae90fc3bff12aebb [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:20.642370 2803 raft_consensus.cc:3037] T a1ab98364a494a76bc429bb90fe513fc P cbf75c267d9a4217bbf9c951246d6a93 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:20.643507 2803 raft_consensus.cc:2454] T a1ab98364a494a76bc429bb90fe513fc P cbf75c267d9a4217bbf9c951246d6a93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27a7405c7485416e8f2b14a5526d9178 in term 1.
I20260808 02:18:20.643537 3254 raft_consensus.cc:2454] T a1ab98364a494a76bc429bb90fe513fc P 053e2be59b2d4d06ae90fc3bff12aebb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27a7405c7485416e8f2b14a5526d9178 in term 1.
I20260808 02:18:20.643832 2919 leader_election.cc:302] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27a7405c7485416e8f2b14a5526d9178, cbf75c267d9a4217bbf9c951246d6a93; no voters:
I20260808 02:18:20.644023 3380 raft_consensus.cc:2781] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:20.644323 3380 raft_consensus.cc:686] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178 [term 1 LEADER]: Becoming Leader. State: Replica: 27a7405c7485416e8f2b14a5526d9178, State: Running, Role: LEADER
I20260808 02:18:20.644735 3380 consensus_queue.cc:237] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e2be59b2d4d06ae90fc3bff12aebb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 35579 } } peers { permanent_uuid: "27a7405c7485416e8f2b14a5526d9178" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41143 } } peers { permanent_uuid: "cbf75c267d9a4217bbf9c951246d6a93" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46067 } }
I20260808 02:18:20.648474 1405 catalog_manager.cc:5495] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27a7405c7485416e8f2b14a5526d9178 (127.24.79.194). New cstate: current_term: 1 leader_uuid: "27a7405c7485416e8f2b14a5526d9178" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053e2be59b2d4d06ae90fc3bff12aebb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 35579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27a7405c7485416e8f2b14a5526d9178" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cbf75c267d9a4217bbf9c951246d6a93" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46067 } health_report { overall_health: UNKNOWN } } }
W20260808 02:18:20.660028 3321 tablet.cc:2369] T a1ab98364a494a76bc429bb90fe513fc P 053e2be59b2d4d06ae90fc3bff12aebb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:18:20.682415 3115 tablet.cc:2369] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:18:20.692624 2878 tablet.cc:2369] T a1ab98364a494a76bc429bb90fe513fc P cbf75c267d9a4217bbf9c951246d6a93: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:18:20.703173 3360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.699238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49292 (local address 127.24.79.194:41143)
0808 02:18:20.699410 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:20.699414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:20.699465 (+ 51us) server_negotiation.cc:407] Connection header received
0808 02:18:20.699620 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:20.699622 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:20.699704 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:20.699817 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:18:20.700044 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.700601 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.701505 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.701758 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.701842 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.702222 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:18:20.702699 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:18:20.702807 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:20.702896 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 02:18:20.703013 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 02:18:20.709645 2803 raft_consensus.cc:1261] T a1ab98364a494a76bc429bb90fe513fc P cbf75c267d9a4217bbf9c951246d6a93 [term 1 FOLLOWER]: Refusing update from remote peer 27a7405c7485416e8f2b14a5526d9178: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:18:20.709661 3254 raft_consensus.cc:1261] T a1ab98364a494a76bc429bb90fe513fc P 053e2be59b2d4d06ae90fc3bff12aebb [term 1 FOLLOWER]: Refusing update from remote peer 27a7405c7485416e8f2b14a5526d9178: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:18:20.710444 3383 consensus_queue.cc:1035] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbf75c267d9a4217bbf9c951246d6a93" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:18:20.710656 3380 consensus_queue.cc:1035] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178 [LEADER]: Connected to new peer: Peer: permanent_uuid: "053e2be59b2d4d06ae90fc3bff12aebb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 35579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:18:20.718611 3387 mvcc.cc:205] Tried to move back new op lower bound from 7316092930898763776 to 7316092930643525632. Current Snapshot: MvccSnapshot[applied={T|T < 7316092930898763776}]
I20260808 02:18:20.726977 3388 mvcc.cc:205] Tried to move back new op lower bound from 7316092930898763776 to 7316092930643525632. Current Snapshot: MvccSnapshot[applied={T|T < 7316092930898763776}]
I20260808 02:18:20.737293 1411 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:18:20.737485 1411 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:18:20.742245 1411 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa78ed5a894742328b0e31dda1f3499a: Sending DeleteTablet for 3 replicas of tablet a1ab98364a494a76bc429bb90fe513fc
I20260808 02:18:20.743785 3227 tablet_service.cc:1514] Processing DeleteTablet for tablet a1ab98364a494a76bc429bb90fe513fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:20 UTC) from {username='oryx', principal='oryx/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:37290
I20260808 02:18:20.743871 3004 tablet_service.cc:1514] Processing DeleteTablet for tablet a1ab98364a494a76bc429bb90fe513fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:20 UTC) from {username='oryx', principal='oryx/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:49278
I20260808 02:18:20.744405 3399 tablet_replica.cc:330] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178: stopping tablet replica
I20260808 02:18:20.744783 3402 tablet_replica.cc:330] T a1ab98364a494a76bc429bb90fe513fc P 053e2be59b2d4d06ae90fc3bff12aebb: stopping tablet replica
I20260808 02:18:20.744746 2765 tablet_service.cc:1514] Processing DeleteTablet for tablet a1ab98364a494a76bc429bb90fe513fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:20 UTC) from {username='oryx', principal='oryx/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:48244
I20260808 02:18:20.744864 3399 raft_consensus.cc:2229] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:18:20.745074 3402 raft_consensus.cc:2229] T a1ab98364a494a76bc429bb90fe513fc P 053e2be59b2d4d06ae90fc3bff12aebb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:18:20.745429 3402 raft_consensus.cc:2258] T a1ab98364a494a76bc429bb90fe513fc P 053e2be59b2d4d06ae90fc3bff12aebb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:18:20.745574 3399 raft_consensus.cc:2258] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:18:20.745847 3403 tablet_replica.cc:330] T a1ab98364a494a76bc429bb90fe513fc P cbf75c267d9a4217bbf9c951246d6a93: stopping tablet replica
I20260808 02:18:20.746130 3403 raft_consensus.cc:2229] T a1ab98364a494a76bc429bb90fe513fc P cbf75c267d9a4217bbf9c951246d6a93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:18:20.746438 3403 raft_consensus.cc:2258] T a1ab98364a494a76bc429bb90fe513fc P cbf75c267d9a4217bbf9c951246d6a93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:18:20.747547 3399 ts_tablet_manager.cc:1905] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:18:20.748579 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 1361
I20260808 02:18:20.755829 3403 ts_tablet_manager.cc:1905] T a1ab98364a494a76bc429bb90fe513fc P cbf75c267d9a4217bbf9c951246d6a93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:18:20.757566 3399 ts_tablet_manager.cc:1918] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:18:20.757704 3399 log.cc:1196] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/a1ab98364a494a76bc429bb90fe513fc
I20260808 02:18:20.758076 3399 ts_tablet_manager.cc:1939] T a1ab98364a494a76bc429bb90fe513fc P 27a7405c7485416e8f2b14a5526d9178: Deleting consensus metadata
I20260808 02:18:20.758883 3403 ts_tablet_manager.cc:1918] T a1ab98364a494a76bc429bb90fe513fc P cbf75c267d9a4217bbf9c951246d6a93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:18:20.759035 3403 log.cc:1196] T a1ab98364a494a76bc429bb90fe513fc P cbf75c267d9a4217bbf9c951246d6a93: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-0/wal/wals/a1ab98364a494a76bc429bb90fe513fc
I20260808 02:18:20.759490 3403 ts_tablet_manager.cc:1939] T a1ab98364a494a76bc429bb90fe513fc P cbf75c267d9a4217bbf9c951246d6a93: Deleting consensus metadata
I20260808 02:18:20.764524 3402 ts_tablet_manager.cc:1905] T a1ab98364a494a76bc429bb90fe513fc P 053e2be59b2d4d06ae90fc3bff12aebb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:18:20.767768 3402 ts_tablet_manager.cc:1918] T a1ab98364a494a76bc429bb90fe513fc P 053e2be59b2d4d06ae90fc3bff12aebb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:18:20.767908 3402 log.cc:1196] T a1ab98364a494a76bc429bb90fe513fc P 053e2be59b2d4d06ae90fc3bff12aebb: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/a1ab98364a494a76bc429bb90fe513fc
I20260808 02:18:20.768262 3402 ts_tablet_manager.cc:1939] T a1ab98364a494a76bc429bb90fe513fc P 053e2be59b2d4d06ae90fc3bff12aebb: Deleting consensus metadata
I20260808 02:18:20.768888 2485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.747113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52430 (local address 127.24.79.253:38013)
0808 02:18:20.747231 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:20.747235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:20.759375 (+ 12140us) server_negotiation.cc:407] Connection header received
0808 02:18:20.759573 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:20.759577 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:20.759640 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:20.759737 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:20.760856 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.761633 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.762767 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.763088 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.763193 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.765414 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:20.765417 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:20.765419 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:20.765452 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:20.766982 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.767453 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.767457 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.767459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.767550 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.767840 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.767844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.767845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.768102 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:20.768242 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:20.768551 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 02:18:20.768720 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 02:18:20.771505 2690 connection.cc:535] server connection from 127.0.0.1:48244 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 02:18:20.772346 2920 connection.cc:535] server connection from 127.0.0.1:49278 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 02:18:20.774065 3147 connection.cc:441] server connection from 127.0.0.1:37290 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37290 (request call id 1) could send its response
I20260808 02:18:20.778553 3337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.746852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39854 (local address 127.24.79.252:44521)
0808 02:18:20.747008 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:20.747013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:20.769077 (+ 22064us) server_negotiation.cc:407] Connection header received
0808 02:18:20.769280 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:20.769284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:20.769348 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:20.769400 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:20.770610 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.771287 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.772137 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.772510 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.772617 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.774559 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:20.774563 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:20.774565 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:20.774596 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:20.776295 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.776758 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.776763 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.776764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.776892 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.777543 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.777547 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.777549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.777804 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:20.777946 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:20.778215 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 02:18:20.778379 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 02:18:20.786051 2485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.777101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52440 (local address 127.24.79.253:38013)
0808 02:18:20.777273 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:20.777277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:20.777662 (+ 385us) server_negotiation.cc:407] Connection header received
0808 02:18:20.777837 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:20.777840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:20.777894 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:20.777961 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:20.779110 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.779741 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.780585 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.780820 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.780914 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.782369 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:20.782374 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:20.782376 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:20.782405 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:20.783988 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.784418 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.784422 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.784424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.784523 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.785016 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.785021 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.785022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.785259 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:20.785370 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:20.785762 (+ 392us) server_negotiation.cc:299] Negotiation successful
0808 02:18:20.785886 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260808 02:18:20.791771 2879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:20.777042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39870 (local address 127.24.79.252:44521)
0808 02:18:20.777179 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:20.777183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:20.783520 (+ 6337us) server_negotiation.cc:407] Connection header received
0808 02:18:20.783623 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:20.783627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:20.783686 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:20.783780 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:20.784547 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.785280 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.786528 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:20.786802 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:20.786905 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:20.788580 (+ 1675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:20.788585 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:20.788588 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:20.788619 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:20.790052 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.790565 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.790569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.790570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.790659 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:20.790951 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:20.790954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:20.790955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:20.791163 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:20.791315 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:20.791521 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 02:18:20.791632 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 02:18:21.722316 3414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:21.721760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.193:53717)
0808 02:18:21.722063 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:21.722192 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":94,"threads_started":1}
W20260808 02:18:21.722680 2876 heartbeater.cc:644] Failed to heartbeat to 127.24.79.254:45167 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
I20260808 02:18:21.726895 3416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:21.725906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.195:50785)
0808 02:18:21.726653 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:21.726754 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":88,"threads_started":1}
W20260808 02:18:21.727308 3317 heartbeater.cc:644] Failed to heartbeat to 127.24.79.254:45167 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
I20260808 02:18:21.727733 3417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:21.727301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.194:56949)
0808 02:18:21.727559 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:21.727638 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":89,"threads_started":1}
W20260808 02:18:21.728066 3112 heartbeater.cc:644] Failed to heartbeat to 127.24.79.254:45167 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
I20260808 02:18:22.243221 3418 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa78ed5a894742328b0e31dda1f3499a)
I20260808 02:18:22.243391 3419 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa78ed5a894742328b0e31dda1f3499a)
I20260808 02:18:22.243405 3418 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: false } }
I20260808 02:18:22.243651 3419 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: false } }
I20260808 02:18:22.243984 3418 leader_election.cc:288] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aa78ed5a894742328b0e31dda1f3499a (127.24.79.254:45167), 5b0b6a0e3064489a90c4859f80edc3f2 (127.24.79.253:38013)
I20260808 02:18:22.244844 3420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:22.244273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.0.0.1:41624)
0808 02:18:22.244614 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:22.244728 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":84,"threads_started":1}
I20260808 02:18:22.244930 3419 leader_election.cc:288] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aa78ed5a894742328b0e31dda1f3499a (127.24.79.254:45167), 44c68eddae7e4a04b957b0ead4437ae8 (127.24.79.252:44521)
W20260808 02:18:22.245440 2568 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
I20260808 02:18:22.245532 3424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:22.245089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.0.0.1:41640)
0808 02:18:22.245368 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:22.245439 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":95,"threads_started":1}
W20260808 02:18:22.245893 2281 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
I20260808 02:18:22.246832 3425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:22.246582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.0.0.1:41658)
0808 02:18:22.246695 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:22.246751 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 02:18:22.246872 3421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:22.246371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.0.0.1:41652)
0808 02:18:22.246743 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:22.246798 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306}
W20260808 02:18:22.247026 2568 leader_election.cc:334] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aa78ed5a894742328b0e31dda1f3499a (127.24.79.254:45167): Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
W20260808 02:18:22.247095 2281 leader_election.cc:334] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aa78ed5a894742328b0e31dda1f3499a (127.24.79.254:45167): Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
I20260808 02:18:22.249246 3420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:22.244699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:38013 (local address 127.0.0.1:52452)
0808 02:18:22.245175 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:22.245189 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:22.245304 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:22.245890 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:22.245893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:22.245914 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:22.246094 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:22.246104 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:22.247121 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:22.247124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:22.248111 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:22.248121 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:22.248804 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:22.248808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:22.248908 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:22.248930 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 02:18:22.248997 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 02:18:22.249128 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":1,"thread_start_us":86,"threads_started":1}
I20260808 02:18:22.249240 3422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:22.244833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52452 (local address 127.24.79.253:38013)
0808 02:18:22.245237 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:22.245242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:22.245260 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:18:22.245472 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:22.245476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:22.245526 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:22.245598 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:22.246218 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:22.246895 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:22.248257 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:22.248687 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:22.248782 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:22.248846 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:22.249026 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 02:18:22.249111 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":65,"threads_started":1}
I20260808 02:18:22.249396 3429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:22.245632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39874 (local address 127.24.79.252:44521)
0808 02:18:22.246054 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:22.246063 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:22.246078 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:18:22.246121 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:22.246124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:22.246169 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:22.246528 (+ 359us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:22.246637 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:22.247375 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:22.248429 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:22.248848 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:22.248941 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:22.248998 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:22.249170 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 02:18:22.249243 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":116,"threads_started":1}
I20260808 02:18:22.249394 3424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:22.245256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:44521 (local address 127.0.0.1:39874)
0808 02:18:22.245672 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:22.245686 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:22.245786 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:22.246301 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:22.246304 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:22.246324 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:22.246424 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:22.246430 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:22.247482 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:22.247486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:22.248301 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:22.248311 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:22.248964 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:22.248968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:22.249057 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:22.249077 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 02:18:22.249152 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 02:18:22.249229 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":65,"threads_started":1}
I20260808 02:18:22.249893 2311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44c68eddae7e4a04b957b0ead4437ae8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" is_pre_election: true
I20260808 02:18:22.249938 2601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "44c68eddae7e4a04b957b0ead4437ae8" is_pre_election: true
I20260808 02:18:22.250061 2311 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44c68eddae7e4a04b957b0ead4437ae8 in term 1.
I20260808 02:18:22.250144 2601 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b0b6a0e3064489a90c4859f80edc3f2 in term 1.
I20260808 02:18:22.250368 2568 leader_election.cc:302] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 44c68eddae7e4a04b957b0ead4437ae8, 5b0b6a0e3064489a90c4859f80edc3f2; no voters: aa78ed5a894742328b0e31dda1f3499a
I20260808 02:18:22.250553 3418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 02:18:22.250540 2279 leader_election.cc:302] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 44c68eddae7e4a04b957b0ead4437ae8, 5b0b6a0e3064489a90c4859f80edc3f2; no voters: aa78ed5a894742328b0e31dda1f3499a
I20260808 02:18:22.250609 3418 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader aa78ed5a894742328b0e31dda1f3499a)
I20260808 02:18:22.250649 3418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:18:22.250774 3419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 02:18:22.250833 3419 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader aa78ed5a894742328b0e31dda1f3499a)
I20260808 02:18:22.250874 3419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:18:22.252115 3419 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: false } }
I20260808 02:18:22.252115 3418 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: false } }
I20260808 02:18:22.252643 3419 leader_election.cc:288] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [CANDIDATE]: Term 2 election: Requested vote from peers aa78ed5a894742328b0e31dda1f3499a (127.24.79.254:45167), 44c68eddae7e4a04b957b0ead4437ae8 (127.24.79.252:44521)
I20260808 02:18:22.252656 3418 leader_election.cc:288] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [CANDIDATE]: Term 2 election: Requested vote from peers aa78ed5a894742328b0e31dda1f3499a (127.24.79.254:45167), 5b0b6a0e3064489a90c4859f80edc3f2 (127.24.79.253:38013)
I20260808 02:18:22.253192 3420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:22.252826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.0.0.1:41668)
0808 02:18:22.253065 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:22.253117 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260808 02:18:22.253202 3424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:22.252815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.0.0.1:41664)
0808 02:18:22.253068 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:22.253130 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
I20260808 02:18:22.253278 2601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "44c68eddae7e4a04b957b0ead4437ae8"
I20260808 02:18:22.253293 2311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44c68eddae7e4a04b957b0ead4437ae8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5b0b6a0e3064489a90c4859f80edc3f2"
I20260808 02:18:22.253440 2601 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b0b6a0e3064489a90c4859f80edc3f2 in current term 2: Already voted for candidate 44c68eddae7e4a04b957b0ead4437ae8 in this term.
I20260808 02:18:22.253463 2311 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 44c68eddae7e4a04b957b0ead4437ae8 in current term 2: Already voted for candidate 5b0b6a0e3064489a90c4859f80edc3f2 in this term.
I20260808 02:18:22.254065 3420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:22.253735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.0.0.1:41680)
0808 02:18:22.253945 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:22.254000 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
W20260808 02:18:22.254199 2568 leader_election.cc:334] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer aa78ed5a894742328b0e31dda1f3499a (127.24.79.254:45167): Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
I20260808 02:18:22.254266 2568 leader_election.cc:302] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 44c68eddae7e4a04b957b0ead4437ae8; no voters: 5b0b6a0e3064489a90c4859f80edc3f2, aa78ed5a894742328b0e31dda1f3499a
I20260808 02:18:22.254319 3424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:22.254098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.0.0.1:41684)
0808 02:18:22.254202 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:22.254255 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 02:18:22.254427 3418 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260808 02:18:22.254441 2281 leader_election.cc:334] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer aa78ed5a894742328b0e31dda1f3499a (127.24.79.254:45167): Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
I20260808 02:18:22.254508 2281 leader_election.cc:302] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b0b6a0e3064489a90c4859f80edc3f2; no voters: 44c68eddae7e4a04b957b0ead4437ae8, aa78ed5a894742328b0e31dda1f3499a
I20260808 02:18:22.254643 3419 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 02:18:22.724027 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:22.723470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.193:36141)
0808 02:18:22.723819 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:22.723902 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":102,"threads_started":1}
I20260808 02:18:22.728358 3433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:22.727877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.195:44575)
0808 02:18:22.728193 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:22.728260 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":137,"threads_started":1}
I20260808 02:18:22.729010 3434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:22.728643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.194:59759)
0808 02:18:22.728872 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:22.728931 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":68,"threads_started":1}
I20260808 02:18:23.725590 3435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:23.725025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.193:58877)
0808 02:18:23.725386 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:23.725471 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":138,"threads_started":1}
I20260808 02:18:23.729568 3436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:23.729105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.195:44041)
0808 02:18:23.729392 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:23.729469 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":89,"threads_started":1}
I20260808 02:18:23.730073 3437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:23.729677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.194:51555)
0808 02:18:23.729943 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:23.729997 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":68,"threads_started":1}
I20260808 02:18:23.801997 3438 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:18:23.802239 3438 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: false } }
I20260808 02:18:23.802738 3438 leader_election.cc:288] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aa78ed5a894742328b0e31dda1f3499a (127.24.79.254:45167), 5b0b6a0e3064489a90c4859f80edc3f2 (127.24.79.253:38013)
I20260808 02:18:23.803473 3440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:23.803041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.0.0.1:41898)
0808 02:18:23.803293 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:23.803370 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":68,"threads_started":1}
I20260808 02:18:23.803766 2311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44c68eddae7e4a04b957b0ead4437ae8" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" is_pre_election: true
I20260808 02:18:23.803936 2311 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44c68eddae7e4a04b957b0ead4437ae8 in term 2.
I20260808 02:18:23.804320 2568 leader_election.cc:302] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44c68eddae7e4a04b957b0ead4437ae8, 5b0b6a0e3064489a90c4859f80edc3f2; no voters:
I20260808 02:18:23.804451 3440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:23.804149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.0.0.1:41906)
0808 02:18:23.804304 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:23.804361 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 02:18:23.804502 3438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260808 02:18:23.804569 3438 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:18:23.804611 3438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 2 FOLLOWER]: Advancing to term 3
W20260808 02:18:23.804632 2568 leader_election.cc:334] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer aa78ed5a894742328b0e31dda1f3499a (127.24.79.254:45167): Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
I20260808 02:18:23.805735 3438 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: false } }
I20260808 02:18:23.806079 3438 leader_election.cc:288] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [CANDIDATE]: Term 3 election: Requested vote from peers aa78ed5a894742328b0e31dda1f3499a (127.24.79.254:45167), 5b0b6a0e3064489a90c4859f80edc3f2 (127.24.79.253:38013)
I20260808 02:18:23.806495 3440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:23.806253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.0.0.1:41918)
0808 02:18:23.806372 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:23.806428 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 02:18:23.806681 2311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44c68eddae7e4a04b957b0ead4437ae8" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5b0b6a0e3064489a90c4859f80edc3f2"
I20260808 02:18:23.806838 2311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 2 FOLLOWER]: Advancing to term 3
I20260808 02:18:23.807284 3440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:23.807031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.0.0.1:41926)
0808 02:18:23.807175 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:23.807226 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 02:18:23.807394 2568 leader_election.cc:334] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer aa78ed5a894742328b0e31dda1f3499a (127.24.79.254:45167): Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
I20260808 02:18:23.808007 2311 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44c68eddae7e4a04b957b0ead4437ae8 in term 3.
I20260808 02:18:23.808342 2568 leader_election.cc:302] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 44c68eddae7e4a04b957b0ead4437ae8, 5b0b6a0e3064489a90c4859f80edc3f2; no voters: aa78ed5a894742328b0e31dda1f3499a
I20260808 02:18:23.808516 3438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 02:18:23.808732 3438 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [term 3 LEADER]: Becoming Leader. State: Replica: 44c68eddae7e4a04b957b0ead4437ae8, State: Running, Role: LEADER
I20260808 02:18:23.809003 3438 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: false } }
I20260808 02:18:23.810379 3442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44c68eddae7e4a04b957b0ead4437ae8. Latest consensus state: current_term: 3 leader_uuid: "44c68eddae7e4a04b957b0ead4437ae8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: false } } }
I20260808 02:18:23.810532 3442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:23.810882 3446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:18:23.813357 3446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:18:23.813968 3446 catalog_manager.cc:1259] Loaded cluster ID: 0a4525e343e5431e9aa0df4f93800b9c
I20260808 02:18:23.814034 3446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:18:23.814663 3446 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:18:23.815197 3446 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8: Loaded TSK: 0
I20260808 02:18:23.815977 3446 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:18:23.883086 2586 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:18:23.886359 2586 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:18:23.889863 3440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:23.889506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.0.0.1:41950)
0808 02:18:23.889687 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:23.889765 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 02:18:23.890229 2311 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [term 3 FOLLOWER]: Refusing update from remote peer 44c68eddae7e4a04b957b0ead4437ae8: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260808 02:18:23.890726 3440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:23.890459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.0.0.1:41960)
0808 02:18:23.890569 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:23.890662 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 02:18:23.890802 3438 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260808 02:18:23.891055 2568 consensus_peers.cc:487] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 -> Peer aa78ed5a894742328b0e31dda1f3499a (127.24.79.254:45167): Couldn't send request to peer aa78ed5a894742328b0e31dda1f3499a. Status: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 02:18:23.892450 3448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44c68eddae7e4a04b957b0ead4437ae8. Latest consensus state: current_term: 3 leader_uuid: "44c68eddae7e4a04b957b0ead4437ae8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: false } } }
I20260808 02:18:23.892575 3448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:18:23.893851 3438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "44c68eddae7e4a04b957b0ead4437ae8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: false } } }
I20260808 02:18:23.893963 3438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:23.894421 3448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "44c68eddae7e4a04b957b0ead4437ae8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa78ed5a894742328b0e31dda1f3499a" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45167 } } peers { permanent_uuid: "5b0b6a0e3064489a90c4859f80edc3f2" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 38013 } attrs { promote: false } } peers { permanent_uuid: "44c68eddae7e4a04b957b0ead4437ae8" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 44521 } attrs { promote: false } } }
I20260808 02:18:23.894550 3448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b0b6a0e3064489a90c4859f80edc3f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:18:23.905521 3440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:23.901309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:41143 (local address 127.0.0.1:49302)
0808 02:18:23.901435 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:23.901450 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:23.901539 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:23.902003 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:23.902006 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:23.902027 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:23.902174 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:23.902181 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:23.903264 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:23.903268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:23.904209 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:23.904221 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:23.905089 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:23.905092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:23.905189 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:23.905218 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 02:18:23.905303 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 02:18:23.905379 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 02:18:23.905551 3460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:23.901451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49302 (local address 127.24.79.194:41143)
0808 02:18:23.901757 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:23.901762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:23.901777 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:18:23.901835 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:23.901838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:23.901891 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:23.901978 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:23.902327 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:23.903138 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:23.904377 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:23.904960 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:23.905069 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:23.905151 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:23.905332 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 02:18:23.905412 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":113,"threads_started":1}
I20260808 02:18:23.906215 3004 tablet_service.cc:1467] Processing CreateTablet for tablet e5ad3c97cfba4a37af88a3412e069422 (DEFAULT_TABLE table=test-table [id=5c5a07fb22a24b448704ab1b1561c2d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:23.906536 3004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5ad3c97cfba4a37af88a3412e069422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:23.907378 3469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:23.901867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:35579 (local address 127.0.0.1:37300)
0808 02:18:23.902802 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:23.902816 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:23.902910 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:23.903339 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:23.903343 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:23.903359 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:23.903445 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:23.903449 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:23.904541 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:23.904545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:23.906172 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:23.906183 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:23.906972 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:23.906976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:23.907065 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:23.907089 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 02:18:23.907153 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 02:18:23.907232 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":882,"mutex_wait_us":1,"thread_start_us":735,"threads_started":1}
I20260808 02:18:23.907410 3465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:23.902159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37300 (local address 127.24.79.195:35579)
0808 02:18:23.902447 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:23.902451 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:23.903019 (+ 568us) server_negotiation.cc:407] Connection header received
0808 02:18:23.903081 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:23.903085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:23.903134 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:23.903207 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:23.903560 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:23.904403 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:23.906299 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:23.906827 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:23.906959 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:23.907026 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:23.907195 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 02:18:23.907282 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":135,"threads_started":1}
I20260808 02:18:23.907603 3462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:23.902133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48246 (local address 127.24.79.193:46067)
0808 02:18:23.902410 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:23.902415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:23.903040 (+ 625us) server_negotiation.cc:407] Connection header received
0808 02:18:23.903232 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:23.903236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:23.903290 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:23.903346 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:23.903709 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:23.904466 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:23.906118 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:23.906709 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:23.906791 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:23.906858 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:23.907273 (+ 415us) server_negotiation.cc:299] Negotiation successful
0808 02:18:23.907378 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":79,"threads_started":1}
I20260808 02:18:23.907972 3468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:23.901896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:46067 (local address 127.0.0.1:48246)
0808 02:18:23.902955 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:23.902968 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:23.903108 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:23.903484 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:23.903487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:23.903503 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:23.903593 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:23.903599 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:23.904574 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:23.904577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:23.905965 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:23.905976 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:23.906989 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:23.906992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:23.907125 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:23.907150 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 02:18:23.907755 (+ 605us) client_negotiation.cc:240] Negotiation successful
0808 02:18:23.907837 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":973,"mutex_wait_us":1,"thread_start_us":719,"threads_started":1}
I20260808 02:18:23.908298 3227 tablet_service.cc:1467] Processing CreateTablet for tablet e5ad3c97cfba4a37af88a3412e069422 (DEFAULT_TABLE table=test-table [id=5c5a07fb22a24b448704ab1b1561c2d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:23.908581 3227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5ad3c97cfba4a37af88a3412e069422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:23.909390 3470 tablet_bootstrap.cc:492] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178: Bootstrap starting.
I20260808 02:18:23.910306 3470 tablet_bootstrap.cc:654] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:23.910985 3473 tablet_bootstrap.cc:492] T e5ad3c97cfba4a37af88a3412e069422 P 053e2be59b2d4d06ae90fc3bff12aebb: Bootstrap starting.
I20260808 02:18:23.911369 2765 tablet_service.cc:1467] Processing CreateTablet for tablet e5ad3c97cfba4a37af88a3412e069422 (DEFAULT_TABLE table=test-table [id=5c5a07fb22a24b448704ab1b1561c2d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:23.911649 2765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5ad3c97cfba4a37af88a3412e069422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:23.912009 3473 tablet_bootstrap.cc:654] T e5ad3c97cfba4a37af88a3412e069422 P 053e2be59b2d4d06ae90fc3bff12aebb: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:23.913134 3473 tablet_bootstrap.cc:492] T e5ad3c97cfba4a37af88a3412e069422 P 053e2be59b2d4d06ae90fc3bff12aebb: No bootstrap required, opened a new log
I20260808 02:18:23.913233 3473 ts_tablet_manager.cc:1397] T e5ad3c97cfba4a37af88a3412e069422 P 053e2be59b2d4d06ae90fc3bff12aebb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:18:23.913765 3473 raft_consensus.cc:348] T e5ad3c97cfba4a37af88a3412e069422 P 053e2be59b2d4d06ae90fc3bff12aebb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a7405c7485416e8f2b14a5526d9178" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41143 } } peers { permanent_uuid: "cbf75c267d9a4217bbf9c951246d6a93" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46067 } } peers { permanent_uuid: "053e2be59b2d4d06ae90fc3bff12aebb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 35579 } }
I20260808 02:18:23.913889 3473 raft_consensus.cc:374] T e5ad3c97cfba4a37af88a3412e069422 P 053e2be59b2d4d06ae90fc3bff12aebb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:23.913923 3473 raft_consensus.cc:729] T e5ad3c97cfba4a37af88a3412e069422 P 053e2be59b2d4d06ae90fc3bff12aebb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 053e2be59b2d4d06ae90fc3bff12aebb, State: Initialized, Role: FOLLOWER
I20260808 02:18:23.914093 3473 consensus_queue.cc:260] T e5ad3c97cfba4a37af88a3412e069422 P 053e2be59b2d4d06ae90fc3bff12aebb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a7405c7485416e8f2b14a5526d9178" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41143 } } peers { permanent_uuid: "cbf75c267d9a4217bbf9c951246d6a93" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46067 } } peers { permanent_uuid: "053e2be59b2d4d06ae90fc3bff12aebb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 35579 } }
I20260808 02:18:23.914213 3475 tablet_bootstrap.cc:492] T e5ad3c97cfba4a37af88a3412e069422 P cbf75c267d9a4217bbf9c951246d6a93: Bootstrap starting.
I20260808 02:18:23.914774 3470 tablet_bootstrap.cc:492] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178: No bootstrap required, opened a new log
I20260808 02:18:23.914877 3470 ts_tablet_manager.cc:1397] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260808 02:18:23.915234 3436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:23.914843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.195:37845)
0808 02:18:23.915068 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:23.915134 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260808 02:18:23.915259 3475 tablet_bootstrap.cc:654] T e5ad3c97cfba4a37af88a3412e069422 P cbf75c267d9a4217bbf9c951246d6a93: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:23.915381 3470 raft_consensus.cc:348] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a7405c7485416e8f2b14a5526d9178" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41143 } } peers { permanent_uuid: "cbf75c267d9a4217bbf9c951246d6a93" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46067 } } peers { permanent_uuid: "053e2be59b2d4d06ae90fc3bff12aebb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 35579 } }
I20260808 02:18:23.915493 3470 raft_consensus.cc:374] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:23.915534 3470 raft_consensus.cc:729] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27a7405c7485416e8f2b14a5526d9178, State: Initialized, Role: FOLLOWER
I20260808 02:18:23.915668 3470 consensus_queue.cc:260] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a7405c7485416e8f2b14a5526d9178" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41143 } } peers { permanent_uuid: "cbf75c267d9a4217bbf9c951246d6a93" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46067 } } peers { permanent_uuid: "053e2be59b2d4d06ae90fc3bff12aebb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 35579 } }
I20260808 02:18:23.916076 3470 ts_tablet_manager.cc:1428] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:18:23.917403 3437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:23.917150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.194:54555)
0808 02:18:23.917260 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:23.917323 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 02:18:23.917979 3111 heartbeater.cc:503] Master 127.24.79.252:44521 was elected leader, sending a full tablet report...
I20260808 02:18:23.918174 3473 ts_tablet_manager.cc:1428] T e5ad3c97cfba4a37af88a3412e069422 P 053e2be59b2d4d06ae90fc3bff12aebb: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260808 02:18:23.918288 3437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:23.918009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.194:34077)
0808 02:18:23.918171 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:23.918224 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260808 02:18:23.918332 3316 heartbeater.cc:503] Master 127.24.79.252:44521 was elected leader, sending a full tablet report...
I20260808 02:18:23.919387 3436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:23.918893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.195:48773)
0808 02:18:23.919253 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:23.919311 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279}
I20260808 02:18:23.921384 3475 tablet_bootstrap.cc:492] T e5ad3c97cfba4a37af88a3412e069422 P cbf75c267d9a4217bbf9c951246d6a93: No bootstrap required, opened a new log
I20260808 02:18:23.921490 3475 ts_tablet_manager.cc:1397] T e5ad3c97cfba4a37af88a3412e069422 P cbf75c267d9a4217bbf9c951246d6a93: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260808 02:18:23.921947 3475 raft_consensus.cc:348] T e5ad3c97cfba4a37af88a3412e069422 P cbf75c267d9a4217bbf9c951246d6a93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a7405c7485416e8f2b14a5526d9178" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41143 } } peers { permanent_uuid: "cbf75c267d9a4217bbf9c951246d6a93" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46067 } } peers { permanent_uuid: "053e2be59b2d4d06ae90fc3bff12aebb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 35579 } }
I20260808 02:18:23.922062 3475 raft_consensus.cc:374] T e5ad3c97cfba4a37af88a3412e069422 P cbf75c267d9a4217bbf9c951246d6a93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:23.922084 3475 raft_consensus.cc:729] T e5ad3c97cfba4a37af88a3412e069422 P cbf75c267d9a4217bbf9c951246d6a93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbf75c267d9a4217bbf9c951246d6a93, State: Initialized, Role: FOLLOWER
I20260808 02:18:23.922200 3475 consensus_queue.cc:260] T e5ad3c97cfba4a37af88a3412e069422 P cbf75c267d9a4217bbf9c951246d6a93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a7405c7485416e8f2b14a5526d9178" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41143 } } peers { permanent_uuid: "cbf75c267d9a4217bbf9c951246d6a93" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46067 } } peers { permanent_uuid: "053e2be59b2d4d06ae90fc3bff12aebb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 35579 } }
I20260808 02:18:23.922533 3475 ts_tablet_manager.cc:1428] T e5ad3c97cfba4a37af88a3412e069422 P cbf75c267d9a4217bbf9c951246d6a93: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 02:18:23.923344 3435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:23.923034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.193:47599)
0808 02:18:23.923171 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:23.923261 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 02:18:23.924069 3435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:23.923824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.193:59297)
0808 02:18:23.923941 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:23.923996 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 02:18:23.924243 2875 heartbeater.cc:503] Master 127.24.79.252:44521 was elected leader, sending a full tablet report...
I20260808 02:18:23.989976 3480 raft_consensus.cc:482] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:18:23.990197 3480 raft_consensus.cc:504] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a7405c7485416e8f2b14a5526d9178" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41143 } } peers { permanent_uuid: "cbf75c267d9a4217bbf9c951246d6a93" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46067 } } peers { permanent_uuid: "053e2be59b2d4d06ae90fc3bff12aebb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 35579 } }
I20260808 02:18:23.990720 3480 leader_election.cc:288] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cbf75c267d9a4217bbf9c951246d6a93 (127.24.79.193:46067), 053e2be59b2d4d06ae90fc3bff12aebb (127.24.79.195:35579)
I20260808 02:18:23.991302 2803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5ad3c97cfba4a37af88a3412e069422" candidate_uuid: "27a7405c7485416e8f2b14a5526d9178" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbf75c267d9a4217bbf9c951246d6a93" is_pre_election: true
I20260808 02:18:23.991328 3254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5ad3c97cfba4a37af88a3412e069422" candidate_uuid: "27a7405c7485416e8f2b14a5526d9178" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "053e2be59b2d4d06ae90fc3bff12aebb" is_pre_election: true
I20260808 02:18:23.991461 2803 raft_consensus.cc:2454] T e5ad3c97cfba4a37af88a3412e069422 P cbf75c267d9a4217bbf9c951246d6a93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27a7405c7485416e8f2b14a5526d9178 in term 0.
I20260808 02:18:23.991469 3254 raft_consensus.cc:2454] T e5ad3c97cfba4a37af88a3412e069422 P 053e2be59b2d4d06ae90fc3bff12aebb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27a7405c7485416e8f2b14a5526d9178 in term 0.
I20260808 02:18:23.991845 2921 leader_election.cc:302] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 053e2be59b2d4d06ae90fc3bff12aebb, 27a7405c7485416e8f2b14a5526d9178, cbf75c267d9a4217bbf9c951246d6a93; no voters:
I20260808 02:18:23.992035 3480 raft_consensus.cc:2781] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:18:23.992110 3480 raft_consensus.cc:482] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:18:23.992146 3480 raft_consensus.cc:3037] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:23.993237 3480 raft_consensus.cc:504] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a7405c7485416e8f2b14a5526d9178" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41143 } } peers { permanent_uuid: "cbf75c267d9a4217bbf9c951246d6a93" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46067 } } peers { permanent_uuid: "053e2be59b2d4d06ae90fc3bff12aebb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 35579 } }
I20260808 02:18:23.993665 3480 leader_election.cc:288] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178 [CANDIDATE]: Term 1 election: Requested vote from peers cbf75c267d9a4217bbf9c951246d6a93 (127.24.79.193:46067), 053e2be59b2d4d06ae90fc3bff12aebb (127.24.79.195:35579)
I20260808 02:18:23.994110 2803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5ad3c97cfba4a37af88a3412e069422" candidate_uuid: "27a7405c7485416e8f2b14a5526d9178" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbf75c267d9a4217bbf9c951246d6a93"
I20260808 02:18:23.994102 3254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5ad3c97cfba4a37af88a3412e069422" candidate_uuid: "27a7405c7485416e8f2b14a5526d9178" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "053e2be59b2d4d06ae90fc3bff12aebb"
I20260808 02:18:23.994251 2803 raft_consensus.cc:3037] T e5ad3c97cfba4a37af88a3412e069422 P cbf75c267d9a4217bbf9c951246d6a93 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:23.994251 3254 raft_consensus.cc:3037] T e5ad3c97cfba4a37af88a3412e069422 P 053e2be59b2d4d06ae90fc3bff12aebb [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:23.995510 3254 raft_consensus.cc:2454] T e5ad3c97cfba4a37af88a3412e069422 P 053e2be59b2d4d06ae90fc3bff12aebb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27a7405c7485416e8f2b14a5526d9178 in term 1.
I20260808 02:18:23.995510 2803 raft_consensus.cc:2454] T e5ad3c97cfba4a37af88a3412e069422 P cbf75c267d9a4217bbf9c951246d6a93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27a7405c7485416e8f2b14a5526d9178 in term 1.
I20260808 02:18:23.995877 2919 leader_election.cc:302] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27a7405c7485416e8f2b14a5526d9178, cbf75c267d9a4217bbf9c951246d6a93; no voters:
I20260808 02:18:23.996091 3480 raft_consensus.cc:2781] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:23.996347 3480 raft_consensus.cc:686] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178 [term 1 LEADER]: Becoming Leader. State: Replica: 27a7405c7485416e8f2b14a5526d9178, State: Running, Role: LEADER
I20260808 02:18:23.996511 3480 consensus_queue.cc:237] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a7405c7485416e8f2b14a5526d9178" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41143 } } peers { permanent_uuid: "cbf75c267d9a4217bbf9c951246d6a93" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46067 } } peers { permanent_uuid: "053e2be59b2d4d06ae90fc3bff12aebb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 35579 } }
I20260808 02:18:23.997298 3437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:23.996986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.194:42305)
0808 02:18:23.997127 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:23.997202 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 02:18:23.998355 2586 catalog_manager.cc:5495] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27a7405c7485416e8f2b14a5526d9178 (127.24.79.194). New cstate: current_term: 1 leader_uuid: "27a7405c7485416e8f2b14a5526d9178" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27a7405c7485416e8f2b14a5526d9178" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 41143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cbf75c267d9a4217bbf9c951246d6a93" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "053e2be59b2d4d06ae90fc3bff12aebb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 35579 } health_report { overall_health: UNKNOWN } } }
I20260808 02:18:24.035399 3460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:24.032321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49304 (local address 127.24.79.194:41143)
0808 02:18:24.032462 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:24.032468 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:24.032485 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:18:24.032541 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:24.032544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:24.032589 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:24.032655 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:18:24.032906 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:24.033471 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:24.034322 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:24.034542 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:24.034636 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:24.034887 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:18:24.035004 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:18:24.035064 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:24.035181 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 02:18:24.035248 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 02:18:24.037889 3254 raft_consensus.cc:1261] T e5ad3c97cfba4a37af88a3412e069422 P 053e2be59b2d4d06ae90fc3bff12aebb [term 1 FOLLOWER]: Refusing update from remote peer 27a7405c7485416e8f2b14a5526d9178: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:18:24.037891 2803 raft_consensus.cc:1261] T e5ad3c97cfba4a37af88a3412e069422 P cbf75c267d9a4217bbf9c951246d6a93 [term 1 FOLLOWER]: Refusing update from remote peer 27a7405c7485416e8f2b14a5526d9178: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:18:24.038437 3480 consensus_queue.cc:1035] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178 [LEADER]: Connected to new peer: Peer: permanent_uuid: "053e2be59b2d4d06ae90fc3bff12aebb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 35579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:18:24.038651 3484 consensus_queue.cc:1035] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbf75c267d9a4217bbf9c951246d6a93" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 46067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:18:24.040537 3436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:24.040207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.195:49681)
0808 02:18:24.040383 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:24.040444 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260808 02:18:24.040977 3435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:24.040670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.193:46607)
0808 02:18:24.040845 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:24.040901 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260808 02:18:24.043708 3437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:24.043446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.194:58819)
0808 02:18:24.043551 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:24.043620 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 02:18:24.046077 3435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:24.045747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.193:54257)
0808 02:18:24.045931 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:24.045992 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260808 02:18:24.047461 2584 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:18:24.047628 2584 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:18:24.049877 3437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:24.049571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.194:59189)
0808 02:18:24.049732 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:24.049793 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260808 02:18:24.050616 3490 mvcc.cc:205] Tried to move back new op lower bound from 7316092944534532096 to 7316092944371421184. Current Snapshot: MvccSnapshot[applied={T|T < 7316092944534532096 or (T in {7316092944534532096})}]
I20260808 02:18:24.051725 3436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:24.051390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45167 (local address 127.24.79.195:46399)
0808 02:18:24.051568 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:24.051644 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:45167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260808 02:18:24.053573 2584 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44c68eddae7e4a04b957b0ead4437ae8: Sending DeleteTablet for 3 replicas of tablet e5ad3c97cfba4a37af88a3412e069422
I20260808 02:18:24.054281 3227 tablet_service.cc:1514] Processing DeleteTablet for tablet e5ad3c97cfba4a37af88a3412e069422 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:24 UTC) from {username='oryx', principal='oryx/127.24.79.252@KRBTEST.COM'} at 127.0.0.1:37300
I20260808 02:18:24.054404 3224 tablet_service.cc:1514] Processing DeleteTablet for tablet e5ad3c97cfba4a37af88a3412e069422 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:24 UTC) from {username='oryx', principal='oryx/127.24.79.252@KRBTEST.COM'} at 127.0.0.1:37300
I20260808 02:18:24.054664 3512 tablet_replica.cc:330] T e5ad3c97cfba4a37af88a3412e069422 P 053e2be59b2d4d06ae90fc3bff12aebb: stopping tablet replica
I20260808 02:18:24.054720 3004 tablet_service.cc:1514] Processing DeleteTablet for tablet e5ad3c97cfba4a37af88a3412e069422 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:24 UTC) from {username='oryx', principal='oryx/127.24.79.252@KRBTEST.COM'} at 127.0.0.1:49302
I20260808 02:18:24.054786 3512 raft_consensus.cc:2229] T e5ad3c97cfba4a37af88a3412e069422 P 053e2be59b2d4d06ae90fc3bff12aebb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:18:24.054777 2765 tablet_service.cc:1514] Processing DeleteTablet for tablet e5ad3c97cfba4a37af88a3412e069422 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:24 UTC) from {username='oryx', principal='oryx/127.24.79.252@KRBTEST.COM'} at 127.0.0.1:48246
I20260808 02:18:24.054946 3512 raft_consensus.cc:2258] T e5ad3c97cfba4a37af88a3412e069422 P 053e2be59b2d4d06ae90fc3bff12aebb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:18:24.055190 3514 tablet_replica.cc:330] T e5ad3c97cfba4a37af88a3412e069422 P cbf75c267d9a4217bbf9c951246d6a93: stopping tablet replica
I20260808 02:18:24.055320 3514 raft_consensus.cc:2229] T e5ad3c97cfba4a37af88a3412e069422 P cbf75c267d9a4217bbf9c951246d6a93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:18:24.055420 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 2518
I20260808 02:18:24.055461 3514 raft_consensus.cc:2258] T e5ad3c97cfba4a37af88a3412e069422 P cbf75c267d9a4217bbf9c951246d6a93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:18:24.056368 3512 ts_tablet_manager.cc:1905] T e5ad3c97cfba4a37af88a3412e069422 P 053e2be59b2d4d06ae90fc3bff12aebb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:18:24.059939 3512 ts_tablet_manager.cc:1918] T e5ad3c97cfba4a37af88a3412e069422 P 053e2be59b2d4d06ae90fc3bff12aebb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:18:24.060209 3513 tablet_replica.cc:330] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178: stopping tablet replica
I20260808 02:18:24.060447 3513 raft_consensus.cc:2229] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:18:24.060640 3512 log.cc:1196] T e5ad3c97cfba4a37af88a3412e069422 P 053e2be59b2d4d06ae90fc3bff12aebb: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/e5ad3c97cfba4a37af88a3412e069422
I20260808 02:18:24.060990 3513 raft_consensus.cc:2258] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:18:24.061403 3512 ts_tablet_manager.cc:1939] T e5ad3c97cfba4a37af88a3412e069422 P 053e2be59b2d4d06ae90fc3bff12aebb: Deleting consensus metadata
I20260808 02:18:24.062436 2568 catalog_manager.cc:4841] TS 053e2be59b2d4d06ae90fc3bff12aebb (127.24.79.195:35579): tablet e5ad3c97cfba4a37af88a3412e069422 (table test-table [id=5c5a07fb22a24b448704ab1b1561c2d5]) successfully deleted
I20260808 02:18:24.062523 3513 ts_tablet_manager.cc:1905] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 02:18:24.062886 2568 catalog_manager.cc:4808] TS 053e2be59b2d4d06ae90fc3bff12aebb (127.24.79.195:35579): delete failed for tablet e5ad3c97cfba4a37af88a3412e069422 because the tablet was not found. No further retry: Not found: Tablet not found: e5ad3c97cfba4a37af88a3412e069422
I20260808 02:18:24.065399 3513 ts_tablet_manager.cc:1918] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:18:24.065510 3513 log.cc:1196] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/e5ad3c97cfba4a37af88a3412e069422
I20260808 02:18:24.065805 3513 ts_tablet_manager.cc:1939] T e5ad3c97cfba4a37af88a3412e069422 P 27a7405c7485416e8f2b14a5526d9178: Deleting consensus metadata
I20260808 02:18:24.066701 2567 catalog_manager.cc:4841] TS 27a7405c7485416e8f2b14a5526d9178 (127.24.79.194:41143): tablet e5ad3c97cfba4a37af88a3412e069422 (table test-table [id=5c5a07fb22a24b448704ab1b1561c2d5]) successfully deleted
I20260808 02:18:24.069655 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 2890
I20260808 02:18:24.070533 3468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:24.069847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:46067 (local address 127.0.0.1:48260)
0808 02:18:24.070351 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:24.070424 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.193:46067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335}
W20260808 02:18:24.076138 2566 catalog_manager.cc:4584] TS cbf75c267d9a4217bbf9c951246d6a93 (127.24.79.193:46067): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e5ad3c97cfba4a37af88a3412e069422: Network error: Client connection negotiation failed: client connection to 127.24.79.193:46067: connect: Connection refused (error 111)
I20260808 02:18:24.088060 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 3121
I20260808 02:18:24.098604 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 2093
I20260808 02:18:24.112250 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 2253
2026-08-08T02:18:24Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7502 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:18:24 dist-test-slave-1vmg krb5kdc[3518](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:18:24 dist-test-slave-1vmg krb5kdc[3518](info): set up 2 sockets
Aug 08 02:18:24 dist-test-slave-1vmg krb5kdc[3518](info): commencing operation
krb5kdc: starting...
W20260808 02:18:25.275236 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:18:25 dist-test-slave-1vmg krb5kdc[3518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:18:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:18:25Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.254.keytab.
Entry for principal oryx/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.254.keytab.
I20260808 02:18:25.431941 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:34655
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.254.keytab
--principal=oryx/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:34655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:25.560578 3539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:25.560925 3539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:25.560966 3539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:25.565783 3539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:18:25.565871 3539 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 02:18:25.565887 3539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:25.565905 3539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:25.565922 3539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:18:25.565935 3539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:18:25.572057 3539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:34655
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.254.keytab
--principal=oryx/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:34655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:25.573302 3539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:25.574443 3539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:25.583971 3546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:25.584008 3548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:25.584043 3550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:25.584417 3539 server_base.cc:1034] running on GCE node
I20260808 02:18:25.585069 3539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:25.586380 3539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:25.587601 3539 hybrid_clock.cc:648] HybridClock initialized: now 1786155505587576 us; error 47 us; skew 500 ppm
Aug 08 02:18:25 dist-test-slave-1vmg krb5kdc[3518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155505, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:25.591128 3539 init.cc:377] Logged in from keytab as oryx/127.24.79.254@KRBTEST.COM (short username oryx)
I20260808 02:18:25.592365 3539 webserver.cc:460] Webserver started at http://127.24.79.254:38701/ using document root <none> and password file <none>
I20260808 02:18:25.593070 3539 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:25.593160 3539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:25.593374 3539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:25.595832 3539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "3d485b7e961b4b41acdb265ce146848c"
format_stamp: "Formatted at 2026-08-08 02:18:25 on dist-test-slave-1vmg"
server_key: "2c9a8934dc8f476b2c1b1848aa4a61ed"
server_key_iv: "4b33240b322a79a0d55e5a04ae7526bd"
server_key_version: "encryptionkey@0"
I20260808 02:18:25.596468 3539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "3d485b7e961b4b41acdb265ce146848c"
format_stamp: "Formatted at 2026-08-08 02:18:25 on dist-test-slave-1vmg"
server_key: "2c9a8934dc8f476b2c1b1848aa4a61ed"
server_key_iv: "4b33240b322a79a0d55e5a04ae7526bd"
server_key_version: "encryptionkey@0"
I20260808 02:18:25.601197 3539 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:18:25.604288 3561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:25.606200 3539 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:18:25.606400 3539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "3d485b7e961b4b41acdb265ce146848c"
format_stamp: "Formatted at 2026-08-08 02:18:25 on dist-test-slave-1vmg"
server_key: "2c9a8934dc8f476b2c1b1848aa4a61ed"
server_key_iv: "4b33240b322a79a0d55e5a04ae7526bd"
server_key_version: "encryptionkey@0"
I20260808 02:18:25.606562 3539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:25.629316 3539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:25.633280 3539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:25.633487 3539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:25.642879 3539 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:34655
I20260808 02:18:25.642887 3654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:34655 every 8 connection(s)
I20260808 02:18:25.644058 3539 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:18:25.647399 3655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:25.651443 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 3539
I20260808 02:18:25.651597 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:18:25.651909 24895 external_mini_cluster.cc:1428] Setting key 06b0a31ef6a56d410631326280604bc7
I20260808 02:18:25.654369 3655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c: Bootstrap starting.
I20260808 02:18:25.656944 3655 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:25.657758 3655 log.cc:824] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:25.660020 3655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c: No bootstrap required, opened a new log
Aug 08 02:18:25 dist-test-slave-1vmg krb5kdc[3518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM
I20260808 02:18:25.664389 3655 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d485b7e961b4b41acdb265ce146848c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 34655 } }
I20260808 02:18:25.664700 3655 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:25.664765 3655 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d485b7e961b4b41acdb265ce146848c, State: Initialized, Role: FOLLOWER
I20260808 02:18:25.665247 3655 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d485b7e961b4b41acdb265ce146848c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 34655 } }
I20260808 02:18:25.665422 3655 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:18:25.665496 3655 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:18:25.665620 3655 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:25.666796 3655 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d485b7e961b4b41acdb265ce146848c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 34655 } }
I20260808 02:18:25.667260 3655 leader_election.cc:302] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d485b7e961b4b41acdb265ce146848c; no voters:
I20260808 02:18:25.667618 3655 leader_election.cc:288] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:18:25.667737 3660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:25.667969 3660 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c [term 1 LEADER]: Becoming Leader. State: Replica: 3d485b7e961b4b41acdb265ce146848c, State: Running, Role: LEADER
I20260808 02:18:25.668413 3660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d485b7e961b4b41acdb265ce146848c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 34655 } }
I20260808 02:18:25.668612 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:25.653710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46040 (local address 127.24.79.254:34655)
0808 02:18:25.654242 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:25.654260 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:25.654323 (+ 63us) server_negotiation.cc:407] Connection header received
0808 02:18:25.655175 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:25.655199 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:25.655589 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:25.655952 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:25.656873 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:25.658028 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:25.658981 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:25.659511 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:25.659689 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:25.662748 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:25.662754 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:25.662779 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:25.662815 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:25.664762 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:25.665319 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:25.665326 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:25.665334 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:25.665513 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:25.665892 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:25.665895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:25.665896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:25.666397 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:25.666573 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:25.667031 (+ 458us) server_negotiation.cc:299] Negotiation successful
0808 02:18:25.667427 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":170,"threads_started":1}
I20260808 02:18:25.669035 3655 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:18:25.669968 3660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d485b7e961b4b41acdb265ce146848c. Latest consensus state: current_term: 1 leader_uuid: "3d485b7e961b4b41acdb265ce146848c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d485b7e961b4b41acdb265ce146848c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 34655 } } }
I20260808 02:18:25.672582 3661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d485b7e961b4b41acdb265ce146848c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d485b7e961b4b41acdb265ce146848c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 34655 } } }
I20260808 02:18:25.672693 3661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:25.670174 3660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c [sys.catalog]: This master's current role is: LEADER
W20260808 02:18:25.674423 3681 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:18:25.674504 3681 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:18:25.674590 3685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:18:25.675387 3685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:18:25.682189 3685 catalog_manager.cc:1347] Generated new cluster ID: 4db756b7701941c0b19c7b1ff47932b7
I20260808 02:18:25.682289 3685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:18:25.695792 3685 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:18:25.697315 3685 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:18:25.705853 3685 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c: Generated new TSK 0
I20260808 02:18:25.706748 3685 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.79.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.79.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.79.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.193.keytab.
Entry for principal oryx/127.24.79.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.193.keytab.
I20260808 02:18:25.749985 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:34655
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.193.keytab
--principal=oryx/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:25.903268 3692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:25.903590 3692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:25.903651 3692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:25.907933 3692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:25.908021 3692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:25.908061 3692 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:18:25.908100 3692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:18:25.913858 3692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.193.keytab
--principal=oryx/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.79.254:34655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:25.915194 3692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:25.916360 3692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:25.927254 3705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:25.927335 3692 server_base.cc:1034] running on GCE node
W20260808 02:18:25.927277 3701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:25.927263 3702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:25.928014 3692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:25.928750 3692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:25.929934 3692 hybrid_clock.cc:648] HybridClock initialized: now 1786155505929899 us; error 52 us; skew 500 ppm
Aug 08 02:18:25 dist-test-slave-1vmg krb5kdc[3518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155505, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:25.933195 3692 init.cc:377] Logged in from keytab as oryx/127.24.79.193@KRBTEST.COM (short username oryx)
I20260808 02:18:25.934278 3692 webserver.cc:460] Webserver started at http://127.24.79.193:34701/ using document root <none> and password file <none>
I20260808 02:18:25.934959 3692 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:25.935051 3692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:25.935308 3692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:25.937615 3692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "7ca6b5f555cd4cfea50c6debdec39870"
format_stamp: "Formatted at 2026-08-08 02:18:25 on dist-test-slave-1vmg"
server_key: "f6206479c1f29cc8b2bcf433e88e3dec"
server_key_iv: "5a6426624036b874c8f62eb74163f173"
server_key_version: "encryptionkey@0"
I20260808 02:18:25.938241 3692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "7ca6b5f555cd4cfea50c6debdec39870"
format_stamp: "Formatted at 2026-08-08 02:18:25 on dist-test-slave-1vmg"
server_key: "f6206479c1f29cc8b2bcf433e88e3dec"
server_key_iv: "5a6426624036b874c8f62eb74163f173"
server_key_version: "encryptionkey@0"
I20260808 02:18:25.942520 3692 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:18:25.945497 3719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:25.947067 3692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:25.947212 3692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "7ca6b5f555cd4cfea50c6debdec39870"
format_stamp: "Formatted at 2026-08-08 02:18:25 on dist-test-slave-1vmg"
server_key: "f6206479c1f29cc8b2bcf433e88e3dec"
server_key_iv: "5a6426624036b874c8f62eb74163f173"
server_key_version: "encryptionkey@0"
I20260808 02:18:25.947337 3692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:25.969354 3692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:25.973523 3692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:25.973770 3692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:25.975575 3692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:25.975647 3692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:25.975715 3692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:25.975740 3692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:18:25 dist-test-slave-1vmg krb5kdc[3518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155505, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM
I20260808 02:18:25.987321 3692 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:43433
I20260808 02:18:25.987443 3895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:43433 every 8 connection(s)
I20260808 02:18:25.988390 3692 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:18:25.990422 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 3692
I20260808 02:18:25.990561 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:18:25.990833 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:25.976503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:35663 (local address 127.24.79.254:34655)
0808 02:18:25.976674 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:25.976679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:25.977667 (+ 988us) server_negotiation.cc:407] Connection header received
0808 02:18:25.978788 (+ 1121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:25.978791 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:25.978857 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:25.978967 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:25.980886 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:25.981493 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:25.982950 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:25.983342 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:25.983478 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:25.986526 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:25.986531 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:25.986534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:25.986569 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:25.988373 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:25.989121 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:25.989126 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:25.989128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:25.989263 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:25.989633 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:25.989637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:25.989639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:25.989855 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:25.989944 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:25.990509 (+ 565us) server_negotiation.cc:299] Negotiation successful
0808 02:18:25.990660 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 02:18:25.990859 24895 external_mini_cluster.cc:1428] Setting key dc0a4e53ebd8b6e29896de19c2a417c6
I20260808 02:18:25.991881 3741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:25.976951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:34655 (local address 127.24.79.193:35663)
0808 02:18:25.977467 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:25.977524 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:25.978489 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:25.979208 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:25.979219 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:25.979750 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:25.980683 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:25.980709 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:25.981678 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:25.981682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:25.982780 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:25.982793 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:25.983551 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:25.983556 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:25.983794 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:25.984531 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:25.984551 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:25.986292 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:25.988530 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:25.988537 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:25.988555 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:25.988966 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:25.989381 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:25.989384 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:25.989385 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:25.989533 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:25.989983 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:25.989989 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:25.990259 (+ 270us) client_negotiation.cc:769] Sending connection context
0808 02:18:25.990443 (+ 184us) client_negotiation.cc:240] Negotiation successful
0808 02:18:25.990816 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":118,"threads_started":1}
I20260808 02:18:25.994988 3897 heartbeater.cc:347] Connected to a master server at 127.24.79.254:34655
I20260808 02:18:25.995191 3897 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:25.995868 3897 heartbeater.cc:511] Master 127.24.79.254:34655 requested a full tablet report, sending...
I20260808 02:18:25.997798 3586 ts_manager.cc:194] Registered new tserver with Master: 7ca6b5f555cd4cfea50c6debdec39870 (127.24.79.193:43433)
I20260808 02:18:25.999562 3586 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.79.193@KRBTEST.COM'} at 127.24.79.193:35663
I20260808 02:18:26.004266 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:25.996328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:39057 (local address 127.24.79.254:34655)
0808 02:18:25.996500 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:25.996504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:25.996663 (+ 159us) server_negotiation.cc:407] Connection header received
0808 02:18:25.996787 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:25.996790 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:25.996834 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:25.996912 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:25.997427 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:25.997993 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:25.998974 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:25.999204 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:25.999300 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.000857 (+ 1557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:26.000861 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:26.000864 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:26.000892 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:26.002468 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:26.002977 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:26.002981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:26.002983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:26.003072 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:26.003359 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:26.003362 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:26.003364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:26.003561 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:26.003670 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:26.003941 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 02:18:26.004083 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":69}
I20260808 02:18:26.004269 3907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:25.996227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:34655 (local address 127.24.79.193:39057)
0808 02:18:25.996571 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:25.996586 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:25.996699 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:25.996953 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:25.996957 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:25.997188 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:25.997306 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:25.997313 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:25.998154 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:25.998157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:25.998838 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:25.998850 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:25.999319 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:25.999322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:25.999421 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.000020 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:26.000033 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:26.000673 (+ 640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:26.002602 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:26.002606 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:26.002608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:26.002843 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:26.003165 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:26.003168 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:26.003171 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:26.003267 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:26.003667 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:26.003670 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:26.003773 (+ 103us) client_negotiation.cc:769] Sending connection context
0808 02:18:26.003866 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 02:18:26.004038 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":147,"threads_started":1}
WARNING: no policy specified for oryx/127.24.79.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.79.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.79.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.194.keytab.
Entry for principal oryx/127.24.79.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.194.keytab.
I20260808 02:18:26.031229 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:34655
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.194.keytab
--principal=oryx/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:26.154191 3909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:26.154485 3909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:26.154531 3909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:26.158731 3909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:26.158835 3909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:26.158890 3909 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:18:26.158986 3909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:18:26.164667 3909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.194.keytab
--principal=oryx/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.79.254:34655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:26.165906 3909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:26.167080 3909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:26.178001 3916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:26.178174 3909 server_base.cc:1034] running on GCE node
W20260808 02:18:26.178009 3915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:26.178375 3920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:26.178841 3909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:26.179605 3909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:26.180789 3909 hybrid_clock.cc:648] HybridClock initialized: now 1786155506180761 us; error 43 us; skew 500 ppm
Aug 08 02:18:26 dist-test-slave-1vmg krb5kdc[3518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155506, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:26.184036 3909 init.cc:377] Logged in from keytab as oryx/127.24.79.194@KRBTEST.COM (short username oryx)
I20260808 02:18:26.185163 3909 webserver.cc:460] Webserver started at http://127.24.79.194:33933/ using document root <none> and password file <none>
I20260808 02:18:26.185810 3909 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:26.185885 3909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:26.186131 3909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:26.188596 3909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "df2742039bb84aa68ee5b5baefa8a161"
format_stamp: "Formatted at 2026-08-08 02:18:26 on dist-test-slave-1vmg"
server_key: "ac108b1d81ed74146c0ff2e016cc646e"
server_key_iv: "eb8109398183e858acfe9581c7e1c5c3"
server_key_version: "encryptionkey@0"
I20260808 02:18:26.189306 3909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "df2742039bb84aa68ee5b5baefa8a161"
format_stamp: "Formatted at 2026-08-08 02:18:26 on dist-test-slave-1vmg"
server_key: "ac108b1d81ed74146c0ff2e016cc646e"
server_key_iv: "eb8109398183e858acfe9581c7e1c5c3"
server_key_version: "encryptionkey@0"
I20260808 02:18:26.194351 3909 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 02:18:26.197741 3928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:26.199719 3909 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260808 02:18:26.199904 3909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "df2742039bb84aa68ee5b5baefa8a161"
format_stamp: "Formatted at 2026-08-08 02:18:26 on dist-test-slave-1vmg"
server_key: "ac108b1d81ed74146c0ff2e016cc646e"
server_key_iv: "eb8109398183e858acfe9581c7e1c5c3"
server_key_version: "encryptionkey@0"
I20260808 02:18:26.200060 3909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:26.216109 3909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:26.220170 3909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:26.220395 3909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:26.222221 3909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:26.222296 3909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:26.222358 3909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:26.222388 3909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:18:26 dist-test-slave-1vmg krb5kdc[3518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155506, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM
I20260808 02:18:26.233819 3909 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:46621
I20260808 02:18:26.233880 4093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:46621 every 8 connection(s)
I20260808 02:18:26.235018 3909 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:18:26.238386 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.223407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:51329 (local address 127.24.79.254:34655)
0808 02:18:26.223543 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:26.223547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:26.224892 (+ 1345us) server_negotiation.cc:407] Connection header received
0808 02:18:26.226112 (+ 1220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:26.226118 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:26.226192 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:26.226306 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:26.227889 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.228463 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.229776 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.230063 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.230180 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.233947 (+ 3767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:26.233951 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:26.233954 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:26.233992 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:26.235870 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:26.236567 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:26.236570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:26.236572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:26.236670 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:26.237123 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:26.237127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:26.237129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:26.237378 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:26.237468 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:26.238112 (+ 644us) server_negotiation.cc:299] Negotiation successful
0808 02:18:26.238249 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 02:18:26.239409 3942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.224066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:34655 (local address 127.24.79.194:51329)
0808 02:18:26.224687 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:26.224741 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:26.225754 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:26.226547 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:26.226559 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:26.227088 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:26.227701 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:26.227718 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.228615 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.228619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:26.229622 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:26.229635 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.230266 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.230270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:26.230453 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.231704 (+ 1251us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:26.231769 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:26.233639 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:26.236034 (+ 2395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:26.236040 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:26.236059 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:26.236412 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:26.236804 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:26.236807 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:26.236808 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:26.236984 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:26.237544 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:26.237551 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:26.237856 (+ 305us) client_negotiation.cc:769] Sending connection context
0808 02:18:26.238111 (+ 255us) client_negotiation.cc:240] Negotiation successful
0808 02:18:26.238440 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":148,"threads_started":1}
I20260808 02:18:26.241020 4096 heartbeater.cc:347] Connected to a master server at 127.24.79.254:34655
I20260808 02:18:26.241151 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 3909
I20260808 02:18:26.241223 4096 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:26.241318 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:18:26.241683 24895 external_mini_cluster.cc:1428] Setting key 863aa137abc75e3e4625d8ca3ce64e44
I20260808 02:18:26.241878 4096 heartbeater.cc:511] Master 127.24.79.254:34655 requested a full tablet report, sending...
I20260808 02:18:26.243188 3586 ts_manager.cc:194] Registered new tserver with Master: df2742039bb84aa68ee5b5baefa8a161 (127.24.79.194:46621)
I20260808 02:18:26.243970 3586 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.79.194@KRBTEST.COM'} at 127.24.79.194:51329
WARNING: no policy specified for oryx/127.24.79.195@KRBTEST.COM; defaulting to no policy
I20260808 02:18:26.254669 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.243914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:37245 (local address 127.24.79.254:34655)
0808 02:18:26.244037 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:26.244042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:26.246111 (+ 2069us) server_negotiation.cc:407] Connection header received
0808 02:18:26.246244 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:26.246248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:26.246299 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:26.246366 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:26.247013 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.247558 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.248692 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.248929 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.249017 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.250597 (+ 1580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:26.250600 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:26.250603 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:26.250634 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:26.252560 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:26.253070 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:26.253074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:26.253076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:26.253191 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:26.253563 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:26.253567 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:26.253569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:26.253908 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:26.254026 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:26.254416 (+ 390us) server_negotiation.cc:299] Negotiation successful
0808 02:18:26.254536 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 02:18:26.254688 4102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.243864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:34655 (local address 127.24.79.194:37245)
0808 02:18:26.245978 (+ 2114us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:26.245996 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:26.246133 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:26.246413 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:26.246416 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:26.246683 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:26.246834 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:26.246842 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.247722 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.247726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:26.248545 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:26.248560 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.249077 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.249082 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:26.249189 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.249866 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:26.249882 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:26.250405 (+ 523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:26.252704 (+ 2299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:26.252707 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:26.252709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:26.252952 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:26.253338 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:26.253341 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:26.253343 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:26.253454 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:26.254101 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:26.254105 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:26.254231 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 02:18:26.254330 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 02:18:26.254481 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1995,"thread_start_us":123,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.79.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.79.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.195.keytab.
Entry for principal oryx/127.24.79.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.195.keytab.
I20260808 02:18:26.280474 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:34655
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.195.keytab
--principal=oryx/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:26.405691 4106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:26.406044 4106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:26.406093 4106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:26.410380 4106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:26.410481 4106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:26.410524 4106 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:18:26.410559 4106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:18:26.416473 4106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.195.keytab
--principal=oryx/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.79.254:34655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:26.417766 4106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:26.418895 4106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:26.429715 4112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:26.429895 4106 server_base.cc:1034] running on GCE node
W20260808 02:18:26.429723 4113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:26.429764 4116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:26.430397 4106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:26.431250 4106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:26.432439 4106 hybrid_clock.cc:648] HybridClock initialized: now 1786155506432406 us; error 52 us; skew 500 ppm
Aug 08 02:18:26 dist-test-slave-1vmg krb5kdc[3518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155506, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:26.435448 4106 init.cc:377] Logged in from keytab as oryx/127.24.79.195@KRBTEST.COM (short username oryx)
I20260808 02:18:26.436488 4106 webserver.cc:460] Webserver started at http://127.24.79.195:38233/ using document root <none> and password file <none>
I20260808 02:18:26.437127 4106 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:26.437222 4106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:26.437438 4106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:26.439805 4106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "5e877cad41a8409fa3f9bf84edfbbc79"
format_stamp: "Formatted at 2026-08-08 02:18:26 on dist-test-slave-1vmg"
server_key: "752287887a2dbfbe31de8c97cb94a2fc"
server_key_iv: "6d228fcf094d554e4c945defecefe1d4"
server_key_version: "encryptionkey@0"
I20260808 02:18:26.440409 4106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "5e877cad41a8409fa3f9bf84edfbbc79"
format_stamp: "Formatted at 2026-08-08 02:18:26 on dist-test-slave-1vmg"
server_key: "752287887a2dbfbe31de8c97cb94a2fc"
server_key_iv: "6d228fcf094d554e4c945defecefe1d4"
server_key_version: "encryptionkey@0"
I20260808 02:18:26.444581 4106 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:18:26.447460 4126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:26.448904 4106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:26.449031 4106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "5e877cad41a8409fa3f9bf84edfbbc79"
format_stamp: "Formatted at 2026-08-08 02:18:26 on dist-test-slave-1vmg"
server_key: "752287887a2dbfbe31de8c97cb94a2fc"
server_key_iv: "6d228fcf094d554e4c945defecefe1d4"
server_key_version: "encryptionkey@0"
I20260808 02:18:26.449149 4106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:26.465163 4106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:26.469089 4106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:26.469306 4106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:26.471287 4106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:26.471367 4106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:26.471424 4106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:26.471458 4106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:18:26 dist-test-slave-1vmg krb5kdc[3518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155506, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM
I20260808 02:18:26.483012 4106 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:46391
I20260808 02:18:26.483425 4286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:46391 every 8 connection(s)
I20260808 02:18:26.484239 4106 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:18:26.487015 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.472062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:37457 (local address 127.24.79.254:34655)
0808 02:18:26.472222 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:26.472228 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:26.473325 (+ 1097us) server_negotiation.cc:407] Connection header received
0808 02:18:26.474678 (+ 1353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:26.474682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:26.474752 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:26.474814 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:26.476913 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.477650 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.479202 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.479492 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.479602 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.482692 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:26.482698 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:26.482701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:26.482744 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:26.484603 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:26.485326 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:26.485331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:26.485332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:26.485449 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:26.485776 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:26.485779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:26.485780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:26.485986 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:26.486066 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:26.486714 (+ 648us) server_negotiation.cc:299] Negotiation successful
0808 02:18:26.486838 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:18:26.487913 4138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.472537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:34655 (local address 127.24.79.195:37457)
0808 02:18:26.473106 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:26.473177 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:26.474353 (+ 1176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:26.475333 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:26.475344 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:26.475897 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:26.476695 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:26.476721 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.477808 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.477812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:26.479050 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:26.479063 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.479685 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.479690 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:26.479910 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.480622 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:26.480647 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:26.482416 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:26.484787 (+ 2371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:26.484794 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:26.484812 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:26.485190 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:26.485541 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:26.485544 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:26.485545 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:26.485697 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:26.486093 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:26.486156 (+ 63us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:26.486447 (+ 291us) client_negotiation.cc:769] Sending connection context
0808 02:18:26.486627 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 02:18:26.486961 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":122,"threads_started":1}
I20260808 02:18:26.489339 4287 heartbeater.cc:347] Connected to a master server at 127.24.79.254:34655
I20260808 02:18:26.489500 4287 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:26.489720 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 4106
I20260808 02:18:26.489858 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:18:26.490125 4287 heartbeater.cc:511] Master 127.24.79.254:34655 requested a full tablet report, sending...
I20260808 02:18:26.490216 24895 external_mini_cluster.cc:1428] Setting key 5f08ada2500795941bf4a6bde1be88d6
I20260808 02:18:26.491357 3586 ts_manager.cc:194] Registered new tserver with Master: 5e877cad41a8409fa3f9bf84edfbbc79 (127.24.79.195:46391)
I20260808 02:18:26.492115 3586 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.79.195@KRBTEST.COM'} at 127.24.79.195:37457
I20260808 02:18:26.492502 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:18:26.500541 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.492054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:34655 (local address 127.24.79.195:40707)
0808 02:18:26.492497 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:26.492513 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:26.492622 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:26.492903 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:26.492906 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:26.493132 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:26.493247 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:26.493254 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.494146 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.494149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:26.494764 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:26.494774 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.495356 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.495360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:26.495441 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.496020 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:26.496033 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:26.496419 (+ 386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:26.498547 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:26.498550 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:26.498552 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:26.498829 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:26.499268 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:26.499271 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:26.499272 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:26.499374 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:26.499943 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:26.499947 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:26.500053 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 02:18:26.500149 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 02:18:26.500330 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":132,"threads_started":1}
I20260808 02:18:26.500535 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.492093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:40707 (local address 127.24.79.254:34655)
0808 02:18:26.492213 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:26.492217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:26.492607 (+ 390us) server_negotiation.cc:407] Connection header received
0808 02:18:26.492755 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:26.492758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:26.492800 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:26.492866 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:26.493388 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.494002 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.494874 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.495211 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.495330 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.496592 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:26.496597 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:26.496628 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:26.496658 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:26.498370 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:26.498985 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:26.498990 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:26.498992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:26.499102 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:26.499478 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:26.499482 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:26.499484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:26.499771 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:26.499890 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:26.500244 (+ 354us) server_negotiation.cc:299] Negotiation successful
0808 02:18:26.500368 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 02:18:26 dist-test-slave-1vmg krb5kdc[3518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155506, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:18:26 dist-test-slave-1vmg krb5kdc[3518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:26 dist-test-slave-1vmg krb5kdc[3518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 02:18:26.522675 3658 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:26.515671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46054 (local address 127.24.79.254:34655)
0808 02:18:26.515841 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:26.515845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:26.515858 (+ 13us) server_negotiation.cc:407] Connection header received
0808 02:18:26.515902 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:26.515905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:26.515944 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:26.516028 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:26.516823 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.517367 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.518259 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.518484 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.518582 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.522524 (+ 3942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46054: BlockingRecv error: recv got EOF from 127.0.0.1:46054 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 08 02:18:26 dist-test-slave-1vmg krb5kdc[3518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155506, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM
I20260808 02:18:26.532809 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.524475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46062 (local address 127.24.79.254:34655)
0808 02:18:26.524641 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:26.524645 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:26.524661 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:18:26.524758 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:26.524760 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:26.524801 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:26.524869 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:26.525480 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.526037 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.526900 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.527214 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.527308 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.529752 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:26.529756 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:26.529758 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:26.529790 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:26.531295 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:26.531694 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:26.531697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:26.531698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:26.531779 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:26.531978 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:26.531980 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:26.531981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:26.532163 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:26.532268 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:26.532534 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 02:18:26.532646 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 02:18:26.536067 3586 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:18:26.539358 3586 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:18:26.556262 4319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.550341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:46391 (local address 127.0.0.1:51318)
0808 02:18:26.550765 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:26.550778 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:26.550871 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:26.551676 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:26.551679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:26.551697 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:26.551798 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:26.551805 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.553435 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.553439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:26.554722 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:26.554733 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.555635 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.555639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:26.555769 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.555817 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 02:18:26.555939 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 02:18:26.556084 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":147,"thread_start_us":92,"threads_started":1}
I20260808 02:18:26.556326 4316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.549732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:43433 (local address 127.0.0.1:57468)
0808 02:18:26.550167 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:26.550208 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:26.550356 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:26.551398 (+ 1042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:26.551405 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:26.551431 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:26.551609 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:26.551616 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.552996 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.553000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:26.554668 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:26.554681 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.555837 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.555841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:26.555988 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.556030 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 02:18:26.556097 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 02:18:26.556175 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":79,"spinlock_wait_cycles":10368,"thread_start_us":105,"threads_started":1}
I20260808 02:18:26.556411 4318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.549744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47988 (local address 127.24.79.194:46621)
0808 02:18:26.551408 (+ 1664us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:26.551420 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:26.551443 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:18:26.551514 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:26.551518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:26.551719 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:26.551919 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:26.552194 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.553387 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.554880 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.555708 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.555888 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.556058 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:26.556176 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 02:18:26.556280 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1549,"thread_start_us":129,"threads_started":1}
I20260808 02:18:26.556438 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.549601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:46621 (local address 127.0.0.1:47988)
0808 02:18:26.550142 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:26.550200 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:26.550357 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:26.551905 (+ 1548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:26.551908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:26.551926 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:26.552029 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:26.552035 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.553489 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.553492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:26.554685 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:26.554693 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.555837 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.555841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:26.556048 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.556071 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:18:26.556140 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 02:18:26.556205 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":80,"spinlock_wait_cycles":4992,"thread_start_us":78,"threads_started":1}
I20260808 02:18:26.556986 4320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.550474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57468 (local address 127.24.79.193:43433)
0808 02:18:26.550750 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:26.550757 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:26.550783 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:18:26.550869 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:26.550875 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:26.551201 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:26.551796 (+ 595us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:26.551965 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.552871 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.554795 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.555682 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.556537 (+ 855us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.556702 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:26.556756 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 02:18:26.556854 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":75,"threads_started":1}
I20260808 02:18:26.557021 4321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.550722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51318 (local address 127.24.79.195:46391)
0808 02:18:26.551191 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:26.551203 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:26.551227 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:18:26.551321 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:26.551327 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:26.551520 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:26.551955 (+ 435us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:26.552129 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.553327 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.554842 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.555501 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.555612 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.555815 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:26.556351 (+ 536us) server_negotiation.cc:299] Negotiation successful
0808 02:18:26.556878 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":172,"threads_started":1}
I20260808 02:18:26.559675 3999 tablet_service.cc:1467] Processing CreateTablet for tablet 083b2390cdfb4da7bb91eec54b68c1a2 (DEFAULT_TABLE table=test-table [id=fb16459b96e3474885e7cae7f1f28362]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:26.559935 4200 tablet_service.cc:1467] Processing CreateTablet for tablet 083b2390cdfb4da7bb91eec54b68c1a2 (DEFAULT_TABLE table=test-table [id=fb16459b96e3474885e7cae7f1f28362]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:26.559934 3797 tablet_service.cc:1467] Processing CreateTablet for tablet 083b2390cdfb4da7bb91eec54b68c1a2 (DEFAULT_TABLE table=test-table [id=fb16459b96e3474885e7cae7f1f28362]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:26.560861 3999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 083b2390cdfb4da7bb91eec54b68c1a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:26.561061 4200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 083b2390cdfb4da7bb91eec54b68c1a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:26.561057 3797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 083b2390cdfb4da7bb91eec54b68c1a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:26.568739 4322 tablet_bootstrap.cc:492] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161: Bootstrap starting.
I20260808 02:18:26.569681 4325 tablet_bootstrap.cc:492] T 083b2390cdfb4da7bb91eec54b68c1a2 P 5e877cad41a8409fa3f9bf84edfbbc79: Bootstrap starting.
I20260808 02:18:26.570346 4324 tablet_bootstrap.cc:492] T 083b2390cdfb4da7bb91eec54b68c1a2 P 7ca6b5f555cd4cfea50c6debdec39870: Bootstrap starting.
I20260808 02:18:26.571529 4322 tablet_bootstrap.cc:654] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:26.571954 4325 tablet_bootstrap.cc:654] T 083b2390cdfb4da7bb91eec54b68c1a2 P 5e877cad41a8409fa3f9bf84edfbbc79: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:26.572345 4322 log.cc:824] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:26.572640 4324 tablet_bootstrap.cc:654] T 083b2390cdfb4da7bb91eec54b68c1a2 P 7ca6b5f555cd4cfea50c6debdec39870: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:26.572817 4325 log.cc:824] T 083b2390cdfb4da7bb91eec54b68c1a2 P 5e877cad41a8409fa3f9bf84edfbbc79: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:26.573463 4324 log.cc:824] T 083b2390cdfb4da7bb91eec54b68c1a2 P 7ca6b5f555cd4cfea50c6debdec39870: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:26.575105 4325 tablet_bootstrap.cc:492] T 083b2390cdfb4da7bb91eec54b68c1a2 P 5e877cad41a8409fa3f9bf84edfbbc79: No bootstrap required, opened a new log
I20260808 02:18:26.575222 4322 tablet_bootstrap.cc:492] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161: No bootstrap required, opened a new log
I20260808 02:18:26.575338 4325 ts_tablet_manager.cc:1397] T 083b2390cdfb4da7bb91eec54b68c1a2 P 5e877cad41a8409fa3f9bf84edfbbc79: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 02:18:26.575438 4324 tablet_bootstrap.cc:492] T 083b2390cdfb4da7bb91eec54b68c1a2 P 7ca6b5f555cd4cfea50c6debdec39870: No bootstrap required, opened a new log
I20260808 02:18:26.575434 4322 ts_tablet_manager.cc:1397] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 02:18:26.575609 4324 ts_tablet_manager.cc:1397] T 083b2390cdfb4da7bb91eec54b68c1a2 P 7ca6b5f555cd4cfea50c6debdec39870: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 02:18:26.578502 4325 raft_consensus.cc:348] T 083b2390cdfb4da7bb91eec54b68c1a2 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:26.578759 4325 raft_consensus.cc:374] T 083b2390cdfb4da7bb91eec54b68c1a2 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:26.578812 4325 raft_consensus.cc:729] T 083b2390cdfb4da7bb91eec54b68c1a2 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e877cad41a8409fa3f9bf84edfbbc79, State: Initialized, Role: FOLLOWER
I20260808 02:18:26.579324 4325 consensus_queue.cc:260] T 083b2390cdfb4da7bb91eec54b68c1a2 P 5e877cad41a8409fa3f9bf84edfbbc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:26.579658 4322 raft_consensus.cc:348] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:26.579960 4322 raft_consensus.cc:374] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:26.580017 4322 raft_consensus.cc:729] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df2742039bb84aa68ee5b5baefa8a161, State: Initialized, Role: FOLLOWER
I20260808 02:18:26.580354 4324 raft_consensus.cc:348] T 083b2390cdfb4da7bb91eec54b68c1a2 P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:26.580493 4322 consensus_queue.cc:260] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:26.580608 4324 raft_consensus.cc:374] T 083b2390cdfb4da7bb91eec54b68c1a2 P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:26.580655 4324 raft_consensus.cc:729] T 083b2390cdfb4da7bb91eec54b68c1a2 P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ca6b5f555cd4cfea50c6debdec39870, State: Initialized, Role: FOLLOWER
I20260808 02:18:26.580745 4325 ts_tablet_manager.cc:1428] T 083b2390cdfb4da7bb91eec54b68c1a2 P 5e877cad41a8409fa3f9bf84edfbbc79: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 02:18:26.581135 4287 heartbeater.cc:503] Master 127.24.79.254:34655 was elected leader, sending a full tablet report...
I20260808 02:18:26.581125 4324 consensus_queue.cc:260] T 083b2390cdfb4da7bb91eec54b68c1a2 P 7ca6b5f555cd4cfea50c6debdec39870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:26.581511 4322 ts_tablet_manager.cc:1428] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 02:18:26.581902 4096 heartbeater.cc:503] Master 127.24.79.254:34655 was elected leader, sending a full tablet report...
I20260808 02:18:26.582033 4324 ts_tablet_manager.cc:1428] T 083b2390cdfb4da7bb91eec54b68c1a2 P 7ca6b5f555cd4cfea50c6debdec39870: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 02:18:26.582394 3897 heartbeater.cc:503] Master 127.24.79.254:34655 was elected leader, sending a full tablet report...
I20260808 02:18:26.594507 4338 raft_consensus.cc:482] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:18:26.594776 4338 raft_consensus.cc:504] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:26.596055 4338 leader_election.cc:288] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ca6b5f555cd4cfea50c6debdec39870 (127.24.79.193:43433), 5e877cad41a8409fa3f9bf84edfbbc79 (127.24.79.195:46391)
I20260808 02:18:26.600723 3942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.596371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:43433 (local address 127.24.79.194:39201)
0808 02:18:26.596507 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:26.596524 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:26.596634 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:26.596906 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:26.596910 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:26.596932 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:26.597065 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:26.597074 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.598042 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.598045 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:26.599315 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:26.599329 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.600242 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.600246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:26.600366 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.600386 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 02:18:26.600457 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 02:18:26.600545 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":24}
I20260808 02:18:26.600710 4320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.596481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:39201 (local address 127.24.79.193:43433)
0808 02:18:26.596631 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:26.596636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:26.596658 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:18:26.596724 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:26.596727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:26.596780 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:26.596876 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:26.597226 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.597922 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.599471 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.600108 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.600240 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.600301 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:26.600490 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 02:18:26.600569 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:18:26.601167 4321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.596849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:50293 (local address 127.24.79.195:46391)
0808 02:18:26.596983 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:26.596987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:26.597330 (+ 343us) server_negotiation.cc:407] Connection header received
0808 02:18:26.597395 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:26.597398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:26.597449 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:26.597545 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:26.597780 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.598363 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.600078 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.600566 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.600683 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.600736 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:26.600958 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 02:18:26.601045 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:18:26.601537 4340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.596675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:46391 (local address 127.24.79.194:50293)
0808 02:18:26.597183 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:26.597197 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:26.597327 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:26.597558 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:26.597560 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:26.597573 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:26.597686 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:26.597691 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.598483 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.598487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:26.599911 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:26.599922 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.600684 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.600688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:26.600801 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.600824 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:18:26.601325 (+ 501us) client_negotiation.cc:240] Negotiation successful
0808 02:18:26.601412 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":1,"thread_start_us":166,"threads_started":1}
I20260808 02:18:26.601507 3828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "083b2390cdfb4da7bb91eec54b68c1a2" candidate_uuid: "df2742039bb84aa68ee5b5baefa8a161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca6b5f555cd4cfea50c6debdec39870" is_pre_election: true
I20260808 02:18:26.601874 3828 raft_consensus.cc:2454] T 083b2390cdfb4da7bb91eec54b68c1a2 P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df2742039bb84aa68ee5b5baefa8a161 in term 0.
I20260808 02:18:26.602180 4228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "083b2390cdfb4da7bb91eec54b68c1a2" candidate_uuid: "df2742039bb84aa68ee5b5baefa8a161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" is_pre_election: true
I20260808 02:18:26.602458 4228 raft_consensus.cc:2454] T 083b2390cdfb4da7bb91eec54b68c1a2 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df2742039bb84aa68ee5b5baefa8a161 in term 0.
I20260808 02:18:26.602439 3931 leader_election.cc:302] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ca6b5f555cd4cfea50c6debdec39870, df2742039bb84aa68ee5b5baefa8a161; no voters:
I20260808 02:18:26.602744 4338 raft_consensus.cc:2781] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:18:26.602831 4338 raft_consensus.cc:482] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:18:26.602882 4338 raft_consensus.cc:3037] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:26.604127 4338 raft_consensus.cc:504] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:26.604521 4338 leader_election.cc:288] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161 [CANDIDATE]: Term 1 election: Requested vote from peers 7ca6b5f555cd4cfea50c6debdec39870 (127.24.79.193:43433), 5e877cad41a8409fa3f9bf84edfbbc79 (127.24.79.195:46391)
I20260808 02:18:26.605072 3828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "083b2390cdfb4da7bb91eec54b68c1a2" candidate_uuid: "df2742039bb84aa68ee5b5baefa8a161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca6b5f555cd4cfea50c6debdec39870"
I20260808 02:18:26.605072 4228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "083b2390cdfb4da7bb91eec54b68c1a2" candidate_uuid: "df2742039bb84aa68ee5b5baefa8a161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e877cad41a8409fa3f9bf84edfbbc79"
I20260808 02:18:26.605232 3828 raft_consensus.cc:3037] T 083b2390cdfb4da7bb91eec54b68c1a2 P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:26.605232 4228 raft_consensus.cc:3037] T 083b2390cdfb4da7bb91eec54b68c1a2 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:26.606823 3828 raft_consensus.cc:2454] T 083b2390cdfb4da7bb91eec54b68c1a2 P 7ca6b5f555cd4cfea50c6debdec39870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df2742039bb84aa68ee5b5baefa8a161 in term 1.
I20260808 02:18:26.606823 4228 raft_consensus.cc:2454] T 083b2390cdfb4da7bb91eec54b68c1a2 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df2742039bb84aa68ee5b5baefa8a161 in term 1.
I20260808 02:18:26.607303 3931 leader_election.cc:302] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e877cad41a8409fa3f9bf84edfbbc79, df2742039bb84aa68ee5b5baefa8a161; no voters:
I20260808 02:18:26.607514 4338 raft_consensus.cc:2781] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:26.607760 4338 raft_consensus.cc:686] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161 [term 1 LEADER]: Becoming Leader. State: Replica: df2742039bb84aa68ee5b5baefa8a161, State: Running, Role: LEADER
I20260808 02:18:26.608129 4338 consensus_queue.cc:237] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:26.611881 3576 catalog_manager.cc:5495] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161 reported cstate change: term changed from 0 to 1, leader changed from <none> to df2742039bb84aa68ee5b5baefa8a161 (127.24.79.194). New cstate: current_term: 1 leader_uuid: "df2742039bb84aa68ee5b5baefa8a161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } health_report { overall_health: UNKNOWN } } }
I20260808 02:18:26.639185 4318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.634994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47992 (local address 127.24.79.194:46621)
0808 02:18:26.635159 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:26.635165 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:26.635183 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:18:26.635317 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:26.635320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:26.635386 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:26.635495 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:18:26.635799 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.636377 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.637383 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.637712 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.637824 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.638159 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:18:26.638685 (+ 526us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:18:26.638811 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:26.638945 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 02:18:26.639021 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 02:18:26.646013 3828 raft_consensus.cc:1261] T 083b2390cdfb4da7bb91eec54b68c1a2 P 7ca6b5f555cd4cfea50c6debdec39870 [term 1 FOLLOWER]: Refusing update from remote peer df2742039bb84aa68ee5b5baefa8a161: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:18:26.646193 4228 raft_consensus.cc:1261] T 083b2390cdfb4da7bb91eec54b68c1a2 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 1 FOLLOWER]: Refusing update from remote peer df2742039bb84aa68ee5b5baefa8a161: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:18:26.646879 4338 consensus_queue.cc:1035] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:18:26.647177 4341 consensus_queue.cc:1035] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:18:26.658169 4345 mvcc.cc:205] Tried to move back new op lower bound from 7316092955213479936 to 7316092955069956096. Current Snapshot: MvccSnapshot[applied={T|T < 7316092955213479936}]
I20260808 02:18:26.660287 4346 mvcc.cc:205] Tried to move back new op lower bound from 7316092955213479936 to 7316092955069956096. Current Snapshot: MvccSnapshot[applied={T|T < 7316092955213479936}]
I20260808 02:18:26.662230 4349 mvcc.cc:205] Tried to move back new op lower bound from 7316092955213479936 to 7316092955069956096. Current Snapshot: MvccSnapshot[applied={T|T < 7316092955213479936}]
I20260808 02:18:26.669553 3576 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:18:26.669762 3576 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:18:26.672858 3576 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c: Sending DeleteTablet for 3 replicas of tablet 083b2390cdfb4da7bb91eec54b68c1a2
I20260808 02:18:26.673903 3999 tablet_service.cc:1514] Processing DeleteTablet for tablet 083b2390cdfb4da7bb91eec54b68c1a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:26 UTC) from {username='oryx', principal='oryx/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:47988
I20260808 02:18:26.673952 4200 tablet_service.cc:1514] Processing DeleteTablet for tablet 083b2390cdfb4da7bb91eec54b68c1a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:26 UTC) from {username='oryx', principal='oryx/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:51318
I20260808 02:18:26.674614 4361 tablet_replica.cc:330] T 083b2390cdfb4da7bb91eec54b68c1a2 P 5e877cad41a8409fa3f9bf84edfbbc79: stopping tablet replica
I20260808 02:18:26.674638 3576 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 02:18:26.674896 3797 tablet_service.cc:1514] Processing DeleteTablet for tablet 083b2390cdfb4da7bb91eec54b68c1a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:26 UTC) from {username='oryx', principal='oryx/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:57468
I20260808 02:18:26.675066 4361 raft_consensus.cc:2229] T 083b2390cdfb4da7bb91eec54b68c1a2 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 02:18:26.675168 3576 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:18:26.675482 4361 raft_consensus.cc:2258] T 083b2390cdfb4da7bb91eec54b68c1a2 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:18:26.676028 4362 tablet_replica.cc:330] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161: stopping tablet replica
I20260808 02:18:26.676389 4362 raft_consensus.cc:2229] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:18:26.676913 4362 raft_consensus.cc:2258] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:18:26.677629 4361 ts_tablet_manager.cc:1905] T 083b2390cdfb4da7bb91eec54b68c1a2 P 5e877cad41a8409fa3f9bf84edfbbc79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:18:26.678512 4362 ts_tablet_manager.cc:1905] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:18:26.680692 4361 ts_tablet_manager.cc:1918] T 083b2390cdfb4da7bb91eec54b68c1a2 P 5e877cad41a8409fa3f9bf84edfbbc79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:18:26.680902 4361 log.cc:1196] T 083b2390cdfb4da7bb91eec54b68c1a2 P 5e877cad41a8409fa3f9bf84edfbbc79: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/083b2390cdfb4da7bb91eec54b68c1a2
I20260808 02:18:26.681298 4361 ts_tablet_manager.cc:1939] T 083b2390cdfb4da7bb91eec54b68c1a2 P 5e877cad41a8409fa3f9bf84edfbbc79: Deleting consensus metadata
I20260808 02:18:26.681409 3999 tablet_service.cc:1467] Processing CreateTablet for tablet 20a7ec22b4d64d2689041315d9035ef5 (DEFAULT_TABLE table=test-table [id=17768a7e0f0b470db5f333013646506d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:26.681738 3999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20a7ec22b4d64d2689041315d9035ef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:26.682036 3797 tablet_service.cc:1467] Processing CreateTablet for tablet 20a7ec22b4d64d2689041315d9035ef5 (DEFAULT_TABLE table=test-table [id=17768a7e0f0b470db5f333013646506d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:26.682353 3797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20a7ec22b4d64d2689041315d9035ef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:26.683223 4200 tablet_service.cc:1467] Processing CreateTablet for tablet 20a7ec22b4d64d2689041315d9035ef5 (DEFAULT_TABLE table=test-table [id=17768a7e0f0b470db5f333013646506d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:26.683480 4200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20a7ec22b4d64d2689041315d9035ef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:26.683461 3565 catalog_manager.cc:4841] TS 5e877cad41a8409fa3f9bf84edfbbc79 (127.24.79.195:46391): tablet 083b2390cdfb4da7bb91eec54b68c1a2 (table test-table [id=fb16459b96e3474885e7cae7f1f28362]) successfully deleted
I20260808 02:18:26.684213 4322 tablet_bootstrap.cc:492] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161: Bootstrap starting.
I20260808 02:18:26.684914 4362 ts_tablet_manager.cc:1918] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:18:26.685022 4362 log.cc:1196] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/083b2390cdfb4da7bb91eec54b68c1a2
I20260808 02:18:26.685232 4322 tablet_bootstrap.cc:654] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:26.685348 4362 ts_tablet_manager.cc:1939] T 083b2390cdfb4da7bb91eec54b68c1a2 P df2742039bb84aa68ee5b5baefa8a161: Deleting consensus metadata
I20260808 02:18:26.685985 4325 tablet_bootstrap.cc:492] T 20a7ec22b4d64d2689041315d9035ef5 P 5e877cad41a8409fa3f9bf84edfbbc79: Bootstrap starting.
I20260808 02:18:26.686713 4322 tablet_bootstrap.cc:492] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161: No bootstrap required, opened a new log
I20260808 02:18:26.686816 4322 ts_tablet_manager.cc:1397] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:18:26.687041 4325 tablet_bootstrap.cc:654] T 20a7ec22b4d64d2689041315d9035ef5 P 5e877cad41a8409fa3f9bf84edfbbc79: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:26.687330 4322 raft_consensus.cc:348] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:26.687440 4322 raft_consensus.cc:374] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:26.687469 4322 raft_consensus.cc:729] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df2742039bb84aa68ee5b5baefa8a161, State: Initialized, Role: FOLLOWER
I20260808 02:18:26.687594 4322 consensus_queue.cc:260] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:26.687832 4324 tablet_bootstrap.cc:492] T 20a7ec22b4d64d2689041315d9035ef5 P 7ca6b5f555cd4cfea50c6debdec39870: Bootstrap starting.
I20260808 02:18:26.687841 4322 ts_tablet_manager.cc:1428] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:18:26.688723 4325 tablet_bootstrap.cc:492] T 20a7ec22b4d64d2689041315d9035ef5 P 5e877cad41a8409fa3f9bf84edfbbc79: No bootstrap required, opened a new log
I20260808 02:18:26.688832 4325 ts_tablet_manager.cc:1397] T 20a7ec22b4d64d2689041315d9035ef5 P 5e877cad41a8409fa3f9bf84edfbbc79: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 02:18:26.688854 4324 tablet_bootstrap.cc:654] T 20a7ec22b4d64d2689041315d9035ef5 P 7ca6b5f555cd4cfea50c6debdec39870: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:26.689354 4325 raft_consensus.cc:348] T 20a7ec22b4d64d2689041315d9035ef5 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:26.689495 4325 raft_consensus.cc:374] T 20a7ec22b4d64d2689041315d9035ef5 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:26.689569 4325 raft_consensus.cc:729] T 20a7ec22b4d64d2689041315d9035ef5 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e877cad41a8409fa3f9bf84edfbbc79, State: Initialized, Role: FOLLOWER
I20260808 02:18:26.689713 4325 consensus_queue.cc:260] T 20a7ec22b4d64d2689041315d9035ef5 P 5e877cad41a8409fa3f9bf84edfbbc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:26.689986 4325 ts_tablet_manager.cc:1428] T 20a7ec22b4d64d2689041315d9035ef5 P 5e877cad41a8409fa3f9bf84edfbbc79: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 02:18:26.691521 4324 tablet_bootstrap.cc:492] T 20a7ec22b4d64d2689041315d9035ef5 P 7ca6b5f555cd4cfea50c6debdec39870: No bootstrap required, opened a new log
I20260808 02:18:26.691627 4324 ts_tablet_manager.cc:1397] T 20a7ec22b4d64d2689041315d9035ef5 P 7ca6b5f555cd4cfea50c6debdec39870: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 02:18:26.692129 4324 raft_consensus.cc:348] T 20a7ec22b4d64d2689041315d9035ef5 P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:26.692246 4324 raft_consensus.cc:374] T 20a7ec22b4d64d2689041315d9035ef5 P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:26.692277 4324 raft_consensus.cc:729] T 20a7ec22b4d64d2689041315d9035ef5 P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ca6b5f555cd4cfea50c6debdec39870, State: Initialized, Role: FOLLOWER
I20260808 02:18:26.692405 4324 consensus_queue.cc:260] T 20a7ec22b4d64d2689041315d9035ef5 P 7ca6b5f555cd4cfea50c6debdec39870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:26.692682 4324 ts_tablet_manager.cc:1428] T 20a7ec22b4d64d2689041315d9035ef5 P 7ca6b5f555cd4cfea50c6debdec39870: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 02:18:26.696069 3562 catalog_manager.cc:4841] TS df2742039bb84aa68ee5b5baefa8a161 (127.24.79.194:46621): tablet 083b2390cdfb4da7bb91eec54b68c1a2 (table test-table [id=fb16459b96e3474885e7cae7f1f28362]) successfully deleted
I20260808 02:18:26.696178 4363 tablet_replica.cc:330] T 083b2390cdfb4da7bb91eec54b68c1a2 P 7ca6b5f555cd4cfea50c6debdec39870: stopping tablet replica
I20260808 02:18:26.696516 4363 raft_consensus.cc:2229] T 083b2390cdfb4da7bb91eec54b68c1a2 P 7ca6b5f555cd4cfea50c6debdec39870 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:18:26.696825 4363 raft_consensus.cc:2258] T 083b2390cdfb4da7bb91eec54b68c1a2 P 7ca6b5f555cd4cfea50c6debdec39870 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:18:26.697329 4356 raft_consensus.cc:482] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:18:26.697438 4356 raft_consensus.cc:504] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:26.697899 4356 leader_election.cc:288] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ca6b5f555cd4cfea50c6debdec39870 (127.24.79.193:43433), 5e877cad41a8409fa3f9bf84edfbbc79 (127.24.79.195:46391)
I20260808 02:18:26.698338 4228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20a7ec22b4d64d2689041315d9035ef5" candidate_uuid: "df2742039bb84aa68ee5b5baefa8a161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" is_pre_election: true
I20260808 02:18:26.698501 4228 raft_consensus.cc:2454] T 20a7ec22b4d64d2689041315d9035ef5 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df2742039bb84aa68ee5b5baefa8a161 in term 0.
I20260808 02:18:26.698825 3931 leader_election.cc:302] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e877cad41a8409fa3f9bf84edfbbc79, df2742039bb84aa68ee5b5baefa8a161; no voters:
I20260808 02:18:26.699081 4356 raft_consensus.cc:2781] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:18:26.699148 4356 raft_consensus.cc:482] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:18:26.699185 4356 raft_consensus.cc:3037] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:26.699194 4363 ts_tablet_manager.cc:1905] T 083b2390cdfb4da7bb91eec54b68c1a2 P 7ca6b5f555cd4cfea50c6debdec39870: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:18:26.699281 3828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20a7ec22b4d64d2689041315d9035ef5" candidate_uuid: "df2742039bb84aa68ee5b5baefa8a161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca6b5f555cd4cfea50c6debdec39870" is_pre_election: true
I20260808 02:18:26.699411 3828 raft_consensus.cc:2454] T 20a7ec22b4d64d2689041315d9035ef5 P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df2742039bb84aa68ee5b5baefa8a161 in term 0.
I20260808 02:18:26.700608 4356 raft_consensus.cc:504] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:26.701021 4356 leader_election.cc:288] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161 [CANDIDATE]: Term 1 election: Requested vote from peers 7ca6b5f555cd4cfea50c6debdec39870 (127.24.79.193:43433), 5e877cad41a8409fa3f9bf84edfbbc79 (127.24.79.195:46391)
I20260808 02:18:26.701440 4228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20a7ec22b4d64d2689041315d9035ef5" candidate_uuid: "df2742039bb84aa68ee5b5baefa8a161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e877cad41a8409fa3f9bf84edfbbc79"
I20260808 02:18:26.701577 4228 raft_consensus.cc:3037] T 20a7ec22b4d64d2689041315d9035ef5 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:26.702483 4363 ts_tablet_manager.cc:1918] T 083b2390cdfb4da7bb91eec54b68c1a2 P 7ca6b5f555cd4cfea50c6debdec39870: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:18:26.702593 4363 log.cc:1196] T 083b2390cdfb4da7bb91eec54b68c1a2 P 7ca6b5f555cd4cfea50c6debdec39870: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-0/wal/wals/083b2390cdfb4da7bb91eec54b68c1a2
I20260808 02:18:26.702791 4228 raft_consensus.cc:2454] T 20a7ec22b4d64d2689041315d9035ef5 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df2742039bb84aa68ee5b5baefa8a161 in term 1.
I20260808 02:18:26.702889 4363 ts_tablet_manager.cc:1939] T 083b2390cdfb4da7bb91eec54b68c1a2 P 7ca6b5f555cd4cfea50c6debdec39870: Deleting consensus metadata
I20260808 02:18:26.703334 3828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20a7ec22b4d64d2689041315d9035ef5" candidate_uuid: "df2742039bb84aa68ee5b5baefa8a161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca6b5f555cd4cfea50c6debdec39870"
I20260808 02:18:26.703464 3828 raft_consensus.cc:3037] T 20a7ec22b4d64d2689041315d9035ef5 P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:26.704403 3565 catalog_manager.cc:4841] TS 7ca6b5f555cd4cfea50c6debdec39870 (127.24.79.193:43433): tablet 083b2390cdfb4da7bb91eec54b68c1a2 (table test-table [id=fb16459b96e3474885e7cae7f1f28362]) successfully deleted
I20260808 02:18:26.704587 3828 raft_consensus.cc:2454] T 20a7ec22b4d64d2689041315d9035ef5 P 7ca6b5f555cd4cfea50c6debdec39870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df2742039bb84aa68ee5b5baefa8a161 in term 1.
I20260808 02:18:26.704945 3931 leader_election.cc:302] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ca6b5f555cd4cfea50c6debdec39870, df2742039bb84aa68ee5b5baefa8a161; no voters:
I20260808 02:18:26.705196 4356 raft_consensus.cc:2781] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:26.705297 4356 raft_consensus.cc:686] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161 [term 1 LEADER]: Becoming Leader. State: Replica: df2742039bb84aa68ee5b5baefa8a161, State: Running, Role: LEADER
I20260808 02:18:26.705442 4356 consensus_queue.cc:237] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:26.707379 3576 catalog_manager.cc:5495] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161 reported cstate change: term changed from 0 to 1, leader changed from <none> to df2742039bb84aa68ee5b5baefa8a161 (127.24.79.194). New cstate: current_term: 1 leader_uuid: "df2742039bb84aa68ee5b5baefa8a161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } health_report { overall_health: UNKNOWN } } }
W20260808 02:18:26.736753 4288 tablet.cc:2369] T 20a7ec22b4d64d2689041315d9035ef5 P 5e877cad41a8409fa3f9bf84edfbbc79: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:18:26.736769 4097 tablet.cc:2369] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:18:26.739974 3898 tablet.cc:2369] T 20a7ec22b4d64d2689041315d9035ef5 P 7ca6b5f555cd4cfea50c6debdec39870: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 02:18:26 dist-test-slave-1vmg krb5kdc[3518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155505, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.254@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM
I20260808 02:18:26.750712 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.740066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46064 (local address 127.24.79.254:34655)
0808 02:18:26.740207 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:26.740213 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:26.740575 (+ 362us) server_negotiation.cc:407] Connection header received
0808 02:18:26.740762 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:26.740765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:26.740809 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:26.740856 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:26.742005 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.742795 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.743922 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.744141 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.744232 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.747019 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:26.747023 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:26.747026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:26.747055 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:26.748793 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:26.749329 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:26.749333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:26.749335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:26.749427 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:26.749682 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:26.749685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:26.749687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:26.749890 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:26.749988 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:26.750435 (+ 447us) server_negotiation.cc:299] Negotiation successful
0808 02:18:26.750541 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 02:18:26.750728 4379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.740050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:34655 (local address 127.0.0.1:46064)
0808 02:18:26.740483 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:26.740499 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:26.740660 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:26.741037 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:26.741039 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:26.741729 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:26.741869 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:26.741879 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.742976 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.742979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:26.743774 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:26.743786 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.744255 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.744258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:26.744337 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.744967 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:26.744980 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:26.746832 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:26.748908 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:26.748922 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:26.748931 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:26.749234 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:26.749511 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:26.749515 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:26.749517 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:26.749600 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:26.750036 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:26.750046 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:26.750186 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 02:18:26.750317 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 02:18:26.750493 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":116,"threads_started":1}
I20260808 02:18:26.756173 3576 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:46064:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 02:18:26.757141 3576 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:18:26.763278 3999 tablet_service.cc:1467] Processing CreateTablet for tablet 8a350ae56e084a4d9630b4f3e12ae6cd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2a88051def5e42a5b66bbee125088c63]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:18:26.763429 4200 tablet_service.cc:1467] Processing CreateTablet for tablet 8a350ae56e084a4d9630b4f3e12ae6cd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2a88051def5e42a5b66bbee125088c63]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:18:26.763506 3797 tablet_service.cc:1467] Processing CreateTablet for tablet 8a350ae56e084a4d9630b4f3e12ae6cd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2a88051def5e42a5b66bbee125088c63]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:18:26.763772 3999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a350ae56e084a4d9630b4f3e12ae6cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:26.763780 4200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a350ae56e084a4d9630b4f3e12ae6cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:26.763922 3797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a350ae56e084a4d9630b4f3e12ae6cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:26.766961 4322 tablet_bootstrap.cc:492] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161: Bootstrap starting.
I20260808 02:18:26.766988 4324 tablet_bootstrap.cc:492] T 8a350ae56e084a4d9630b4f3e12ae6cd P 7ca6b5f555cd4cfea50c6debdec39870: Bootstrap starting.
I20260808 02:18:26.767019 4325 tablet_bootstrap.cc:492] T 8a350ae56e084a4d9630b4f3e12ae6cd P 5e877cad41a8409fa3f9bf84edfbbc79: Bootstrap starting.
I20260808 02:18:26.768004 4325 tablet_bootstrap.cc:654] T 8a350ae56e084a4d9630b4f3e12ae6cd P 5e877cad41a8409fa3f9bf84edfbbc79: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:26.768004 4322 tablet_bootstrap.cc:654] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:26.768036 4324 tablet_bootstrap.cc:654] T 8a350ae56e084a4d9630b4f3e12ae6cd P 7ca6b5f555cd4cfea50c6debdec39870: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:26.769613 4324 tablet_bootstrap.cc:492] T 8a350ae56e084a4d9630b4f3e12ae6cd P 7ca6b5f555cd4cfea50c6debdec39870: No bootstrap required, opened a new log
I20260808 02:18:26.769709 4324 ts_tablet_manager.cc:1397] T 8a350ae56e084a4d9630b4f3e12ae6cd P 7ca6b5f555cd4cfea50c6debdec39870: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:18:26.770242 4324 raft_consensus.cc:348] T 8a350ae56e084a4d9630b4f3e12ae6cd P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } }
I20260808 02:18:26.770356 4324 raft_consensus.cc:374] T 8a350ae56e084a4d9630b4f3e12ae6cd P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:26.770385 4324 raft_consensus.cc:729] T 8a350ae56e084a4d9630b4f3e12ae6cd P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ca6b5f555cd4cfea50c6debdec39870, State: Initialized, Role: FOLLOWER
I20260808 02:18:26.770519 4324 consensus_queue.cc:260] T 8a350ae56e084a4d9630b4f3e12ae6cd P 7ca6b5f555cd4cfea50c6debdec39870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } }
I20260808 02:18:26.770798 4324 ts_tablet_manager.cc:1428] T 8a350ae56e084a4d9630b4f3e12ae6cd P 7ca6b5f555cd4cfea50c6debdec39870: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:18:26.771181 4339 tablet_replica.cc:439] T 8a350ae56e084a4d9630b4f3e12ae6cd P 7ca6b5f555cd4cfea50c6debdec39870: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } }
I20260808 02:18:26.771327 4339 tablet_replica.cc:442] T 8a350ae56e084a4d9630b4f3e12ae6cd P 7ca6b5f555cd4cfea50c6debdec39870: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:18:26.771731 4325 tablet_bootstrap.cc:492] T 8a350ae56e084a4d9630b4f3e12ae6cd P 5e877cad41a8409fa3f9bf84edfbbc79: No bootstrap required, opened a new log
I20260808 02:18:26.771828 4325 ts_tablet_manager.cc:1397] T 8a350ae56e084a4d9630b4f3e12ae6cd P 5e877cad41a8409fa3f9bf84edfbbc79: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260808 02:18:26.772377 4325 raft_consensus.cc:348] T 8a350ae56e084a4d9630b4f3e12ae6cd P 5e877cad41a8409fa3f9bf84edfbbc79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } }
I20260808 02:18:26.772500 4325 raft_consensus.cc:374] T 8a350ae56e084a4d9630b4f3e12ae6cd P 5e877cad41a8409fa3f9bf84edfbbc79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:26.772531 4325 raft_consensus.cc:729] T 8a350ae56e084a4d9630b4f3e12ae6cd P 5e877cad41a8409fa3f9bf84edfbbc79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e877cad41a8409fa3f9bf84edfbbc79, State: Initialized, Role: FOLLOWER
I20260808 02:18:26.772678 4325 consensus_queue.cc:260] T 8a350ae56e084a4d9630b4f3e12ae6cd P 5e877cad41a8409fa3f9bf84edfbbc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } }
I20260808 02:18:26.772956 4325 ts_tablet_manager.cc:1428] T 8a350ae56e084a4d9630b4f3e12ae6cd P 5e877cad41a8409fa3f9bf84edfbbc79: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:18:26.773432 4337 tablet_replica.cc:439] T 8a350ae56e084a4d9630b4f3e12ae6cd P 5e877cad41a8409fa3f9bf84edfbbc79: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } }
I20260808 02:18:26.773633 4337 tablet_replica.cc:442] T 8a350ae56e084a4d9630b4f3e12ae6cd P 5e877cad41a8409fa3f9bf84edfbbc79: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:18:26.775590 4322 tablet_bootstrap.cc:492] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161: No bootstrap required, opened a new log
I20260808 02:18:26.775698 4322 ts_tablet_manager.cc:1397] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260808 02:18:26.776172 4322 raft_consensus.cc:348] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } }
I20260808 02:18:26.776281 4322 raft_consensus.cc:374] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:26.776312 4322 raft_consensus.cc:729] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df2742039bb84aa68ee5b5baefa8a161, State: Initialized, Role: FOLLOWER
I20260808 02:18:26.776449 4322 consensus_queue.cc:260] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } }
I20260808 02:18:26.776742 4322 ts_tablet_manager.cc:1428] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:18:26.776706 4356 tablet_replica.cc:439] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } }
I20260808 02:18:26.776834 4356 tablet_replica.cc:442] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:18:26.805820 4356 raft_consensus.cc:482] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:18:26.805997 4356 raft_consensus.cc:504] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } }
I20260808 02:18:26.806509 4356 leader_election.cc:288] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e877cad41a8409fa3f9bf84edfbbc79 (127.24.79.195:46391), 7ca6b5f555cd4cfea50c6debdec39870 (127.24.79.193:43433)
I20260808 02:18:26.807183 4228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a350ae56e084a4d9630b4f3e12ae6cd" candidate_uuid: "df2742039bb84aa68ee5b5baefa8a161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" is_pre_election: true
I20260808 02:18:26.807221 3828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a350ae56e084a4d9630b4f3e12ae6cd" candidate_uuid: "df2742039bb84aa68ee5b5baefa8a161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca6b5f555cd4cfea50c6debdec39870" is_pre_election: true
I20260808 02:18:26.807368 4228 raft_consensus.cc:2454] T 8a350ae56e084a4d9630b4f3e12ae6cd P 5e877cad41a8409fa3f9bf84edfbbc79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df2742039bb84aa68ee5b5baefa8a161 in term 0.
I20260808 02:18:26.807413 3828 raft_consensus.cc:2454] T 8a350ae56e084a4d9630b4f3e12ae6cd P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df2742039bb84aa68ee5b5baefa8a161 in term 0.
I20260808 02:18:26.807739 3931 leader_election.cc:302] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e877cad41a8409fa3f9bf84edfbbc79, df2742039bb84aa68ee5b5baefa8a161; no voters:
I20260808 02:18:26.807966 4356 raft_consensus.cc:2781] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:18:26.808045 4356 raft_consensus.cc:482] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:18:26.808095 4356 raft_consensus.cc:3037] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:26.809584 4356 raft_consensus.cc:504] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } }
I20260808 02:18:26.810032 4356 leader_election.cc:288] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161 [CANDIDATE]: Term 1 election: Requested vote from peers 5e877cad41a8409fa3f9bf84edfbbc79 (127.24.79.195:46391), 7ca6b5f555cd4cfea50c6debdec39870 (127.24.79.193:43433)
I20260808 02:18:26.810585 4228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a350ae56e084a4d9630b4f3e12ae6cd" candidate_uuid: "df2742039bb84aa68ee5b5baefa8a161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e877cad41a8409fa3f9bf84edfbbc79"
I20260808 02:18:26.810746 4228 raft_consensus.cc:3037] T 8a350ae56e084a4d9630b4f3e12ae6cd P 5e877cad41a8409fa3f9bf84edfbbc79 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:26.810684 3828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a350ae56e084a4d9630b4f3e12ae6cd" candidate_uuid: "df2742039bb84aa68ee5b5baefa8a161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca6b5f555cd4cfea50c6debdec39870"
I20260808 02:18:26.810829 3828 raft_consensus.cc:3037] T 8a350ae56e084a4d9630b4f3e12ae6cd P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:26.812050 3828 raft_consensus.cc:2454] T 8a350ae56e084a4d9630b4f3e12ae6cd P 7ca6b5f555cd4cfea50c6debdec39870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df2742039bb84aa68ee5b5baefa8a161 in term 1.
I20260808 02:18:26.812041 4228 raft_consensus.cc:2454] T 8a350ae56e084a4d9630b4f3e12ae6cd P 5e877cad41a8409fa3f9bf84edfbbc79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df2742039bb84aa68ee5b5baefa8a161 in term 1.
I20260808 02:18:26.812420 3931 leader_election.cc:302] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ca6b5f555cd4cfea50c6debdec39870, df2742039bb84aa68ee5b5baefa8a161; no voters:
I20260808 02:18:26.812614 4356 raft_consensus.cc:2781] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:26.812738 4356 raft_consensus.cc:686] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161 [term 1 LEADER]: Becoming Leader. State: Replica: df2742039bb84aa68ee5b5baefa8a161, State: Running, Role: LEADER
I20260808 02:18:26.812898 4356 consensus_queue.cc:237] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } }
I20260808 02:18:26.814013 4341 tablet_replica.cc:439] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161: TxnStatusTablet state changed. Reason: New leader df2742039bb84aa68ee5b5baefa8a161. Latest consensus state: current_term: 1 leader_uuid: "df2742039bb84aa68ee5b5baefa8a161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } }
I20260808 02:18:26.814267 4341 tablet_replica.cc:442] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:18:26.814656 4386 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 02:18:26.814855 4386 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 02:18:26.815083 3585 catalog_manager.cc:5495] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161 reported cstate change: term changed from 0 to 1, leader changed from <none> to df2742039bb84aa68ee5b5baefa8a161 (127.24.79.194). New cstate: current_term: 1 leader_uuid: "df2742039bb84aa68ee5b5baefa8a161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } health_report { overall_health: UNKNOWN } } }
I20260808 02:18:26.831054 4379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.826996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:46621 (local address 127.0.0.1:48004)
0808 02:18:26.827310 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:26.827323 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:26.827434 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:26.827734 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:26.827736 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:26.827743 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:18:26.827861 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:26.827865 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.828904 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.828907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:26.829539 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:26.829548 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.830150 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.830153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:26.830269 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.830328 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:18:26.830674 (+ 346us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:18:26.830697 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:18:26.830763 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 02:18:26.830853 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207}
I20260808 02:18:26.831111 4318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.827132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48004 (local address 127.24.79.194:46621)
0808 02:18:26.827292 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:26.827297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:26.827373 (+ 76us) server_negotiation.cc:407] Connection header received
0808 02:18:26.827552 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:26.827555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:26.827620 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:26.827723 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:18:26.827965 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.828741 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.829669 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.830014 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.830134 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.830471 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:18:26.830574 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:18:26.830655 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:26.830785 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 02:18:26.830850 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:18:26.834332 3828 raft_consensus.cc:1261] T 8a350ae56e084a4d9630b4f3e12ae6cd P 7ca6b5f555cd4cfea50c6debdec39870 [term 1 FOLLOWER]: Refusing update from remote peer df2742039bb84aa68ee5b5baefa8a161: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:18:26.834460 4228 raft_consensus.cc:1261] T 8a350ae56e084a4d9630b4f3e12ae6cd P 5e877cad41a8409fa3f9bf84edfbbc79 [term 1 FOLLOWER]: Refusing update from remote peer df2742039bb84aa68ee5b5baefa8a161: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:18:26.834849 4341 consensus_queue.cc:1035] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:18:26.835060 4356 consensus_queue.cc:1035] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:18:26.836618 4339 tablet_replica.cc:439] T 8a350ae56e084a4d9630b4f3e12ae6cd P 7ca6b5f555cd4cfea50c6debdec39870: TxnStatusTablet state changed. Reason: New leader df2742039bb84aa68ee5b5baefa8a161. Latest consensus state: current_term: 1 leader_uuid: "df2742039bb84aa68ee5b5baefa8a161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } }
I20260808 02:18:26.836748 4339 tablet_replica.cc:442] T 8a350ae56e084a4d9630b4f3e12ae6cd P 7ca6b5f555cd4cfea50c6debdec39870: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:18:26.837335 4337 tablet_replica.cc:439] T 8a350ae56e084a4d9630b4f3e12ae6cd P 5e877cad41a8409fa3f9bf84edfbbc79: TxnStatusTablet state changed. Reason: New leader df2742039bb84aa68ee5b5baefa8a161. Latest consensus state: current_term: 1 leader_uuid: "df2742039bb84aa68ee5b5baefa8a161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } }
I20260808 02:18:26.837463 4337 tablet_replica.cc:442] T 8a350ae56e084a4d9630b4f3e12ae6cd P 5e877cad41a8409fa3f9bf84edfbbc79: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:18:26.838512 4341 tablet_replica.cc:439] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "df2742039bb84aa68ee5b5baefa8a161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } }
I20260808 02:18:26.838636 4341 tablet_replica.cc:442] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:18:26.841266 4338 tablet_replica.cc:439] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "df2742039bb84aa68ee5b5baefa8a161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } }
I20260808 02:18:26.841436 4338 tablet_replica.cc:442] T 8a350ae56e084a4d9630b4f3e12ae6cd P df2742039bb84aa68ee5b5baefa8a161: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:18:26.842620 4337 tablet_replica.cc:439] T 8a350ae56e084a4d9630b4f3e12ae6cd P 5e877cad41a8409fa3f9bf84edfbbc79: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "df2742039bb84aa68ee5b5baefa8a161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } }
I20260808 02:18:26.842756 4337 tablet_replica.cc:442] T 8a350ae56e084a4d9630b4f3e12ae6cd P 5e877cad41a8409fa3f9bf84edfbbc79: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:18:26.842989 4339 tablet_replica.cc:439] T 8a350ae56e084a4d9630b4f3e12ae6cd P 7ca6b5f555cd4cfea50c6debdec39870: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "df2742039bb84aa68ee5b5baefa8a161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } } peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } }
I20260808 02:18:26.843119 4339 tablet_replica.cc:442] T 8a350ae56e084a4d9630b4f3e12ae6cd P 7ca6b5f555cd4cfea50c6debdec39870: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:18:26.856375 4394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.852718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:46621 (local address 127.24.79.194:55691)
0808 02:18:26.853035 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:26.853047 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:26.853241 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:26.853498 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:26.853501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:26.853508 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:18:26.853613 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:26.853620 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.854386 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.854390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:26.855091 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:26.855102 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.855555 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.855559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:26.855637 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.855692 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:18:26.856043 (+ 351us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:18:26.856049 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 02:18:26.856107 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 02:18:26.856211 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":1,"thread_start_us":120,"threads_started":1}
I20260808 02:18:26.856375 4318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.852789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:55691 (local address 127.24.79.194:46621)
0808 02:18:26.852940 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:26.852944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:26.853127 (+ 183us) server_negotiation.cc:407] Connection header received
0808 02:18:26.853356 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:26.853379 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:26.853422 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:26.853478 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:18:26.853709 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.854287 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.855221 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.855442 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.855534 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.855874 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:18:26.855962 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:18:26.856021 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:26.856124 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 02:18:26.856189 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":69}
I20260808 02:18:26.864969 4228 raft_consensus.cc:1261] T 20a7ec22b4d64d2689041315d9035ef5 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 1 FOLLOWER]: Refusing update from remote peer df2742039bb84aa68ee5b5baefa8a161: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:18:26.865023 3828 raft_consensus.cc:1261] T 20a7ec22b4d64d2689041315d9035ef5 P 7ca6b5f555cd4cfea50c6debdec39870 [term 1 FOLLOWER]: Refusing update from remote peer df2742039bb84aa68ee5b5baefa8a161: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:18:26.865520 4338 consensus_queue.cc:1035] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:18:26.865708 4356 consensus_queue.cc:1035] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:18:26.918668 3585 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:18:26.918848 3585 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:18:26.921361 3585 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c: Sending DeleteTablet for 3 replicas of tablet 20a7ec22b4d64d2689041315d9035ef5
I20260808 02:18:26.922153 3997 tablet_service.cc:1514] Processing DeleteTablet for tablet 20a7ec22b4d64d2689041315d9035ef5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:26 UTC) from {username='oryx', principal='oryx/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:47988
I20260808 02:18:26.922279 3797 tablet_service.cc:1514] Processing DeleteTablet for tablet 20a7ec22b4d64d2689041315d9035ef5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:26 UTC) from {username='oryx', principal='oryx/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:57468
I20260808 02:18:26.922422 4200 tablet_service.cc:1514] Processing DeleteTablet for tablet 20a7ec22b4d64d2689041315d9035ef5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:26 UTC) from {username='oryx', principal='oryx/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:51318
I20260808 02:18:26.922523 4363 tablet_replica.cc:330] T 20a7ec22b4d64d2689041315d9035ef5 P 7ca6b5f555cd4cfea50c6debdec39870: stopping tablet replica
I20260808 02:18:26.922657 4362 tablet_replica.cc:330] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161: stopping tablet replica
I20260808 02:18:26.922653 4361 tablet_replica.cc:330] T 20a7ec22b4d64d2689041315d9035ef5 P 5e877cad41a8409fa3f9bf84edfbbc79: stopping tablet replica
I20260808 02:18:26.922654 4363 raft_consensus.cc:2229] T 20a7ec22b4d64d2689041315d9035ef5 P 7ca6b5f555cd4cfea50c6debdec39870 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:18:26.922791 4363 raft_consensus.cc:2258] T 20a7ec22b4d64d2689041315d9035ef5 P 7ca6b5f555cd4cfea50c6debdec39870 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:18:26.922791 4361 raft_consensus.cc:2229] T 20a7ec22b4d64d2689041315d9035ef5 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:18:26.922796 4362 raft_consensus.cc:2229] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:18:26.922945 4361 raft_consensus.cc:2258] T 20a7ec22b4d64d2689041315d9035ef5 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:18:26.923049 4362 raft_consensus.cc:2258] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:18:26.923970 4361 ts_tablet_manager.cc:1905] T 20a7ec22b4d64d2689041315d9035ef5 P 5e877cad41a8409fa3f9bf84edfbbc79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:18:26.923990 4362 ts_tablet_manager.cc:1905] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:18:26.926762 4361 ts_tablet_manager.cc:1918] T 20a7ec22b4d64d2689041315d9035ef5 P 5e877cad41a8409fa3f9bf84edfbbc79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:18:26.926854 4361 log.cc:1196] T 20a7ec22b4d64d2689041315d9035ef5 P 5e877cad41a8409fa3f9bf84edfbbc79: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/20a7ec22b4d64d2689041315d9035ef5
I20260808 02:18:26.927111 4362 ts_tablet_manager.cc:1918] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:18:26.927274 4361 ts_tablet_manager.cc:1939] T 20a7ec22b4d64d2689041315d9035ef5 P 5e877cad41a8409fa3f9bf84edfbbc79: Deleting consensus metadata
I20260808 02:18:26.927274 4362 log.cc:1196] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/20a7ec22b4d64d2689041315d9035ef5
I20260808 02:18:26.927610 4362 ts_tablet_manager.cc:1939] T 20a7ec22b4d64d2689041315d9035ef5 P df2742039bb84aa68ee5b5baefa8a161: Deleting consensus metadata
I20260808 02:18:26.928427 4363 ts_tablet_manager.cc:1905] T 20a7ec22b4d64d2689041315d9035ef5 P 7ca6b5f555cd4cfea50c6debdec39870: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:18:26.928773 3562 catalog_manager.cc:4841] TS df2742039bb84aa68ee5b5baefa8a161 (127.24.79.194:46621): tablet 20a7ec22b4d64d2689041315d9035ef5 (table test-table [id=17768a7e0f0b470db5f333013646506d]) successfully deleted
I20260808 02:18:26.934089 3565 catalog_manager.cc:4841] TS 5e877cad41a8409fa3f9bf84edfbbc79 (127.24.79.195:46391): tablet 20a7ec22b4d64d2689041315d9035ef5 (table test-table [id=17768a7e0f0b470db5f333013646506d]) successfully deleted
I20260808 02:18:26.934854 4363 ts_tablet_manager.cc:1918] T 20a7ec22b4d64d2689041315d9035ef5 P 7ca6b5f555cd4cfea50c6debdec39870: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:18:26.935003 4363 log.cc:1196] T 20a7ec22b4d64d2689041315d9035ef5 P 7ca6b5f555cd4cfea50c6debdec39870: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-0/wal/wals/20a7ec22b4d64d2689041315d9035ef5
I20260808 02:18:26.935297 4363 ts_tablet_manager.cc:1939] T 20a7ec22b4d64d2689041315d9035ef5 P 7ca6b5f555cd4cfea50c6debdec39870: Deleting consensus metadata
I20260808 02:18:26.936334 3565 catalog_manager.cc:4841] TS 7ca6b5f555cd4cfea50c6debdec39870 (127.24.79.193:43433): tablet 20a7ec22b4d64d2689041315d9035ef5 (table test-table [id=17768a7e0f0b470db5f333013646506d]) successfully deleted
Aug 08 02:18:26 dist-test-slave-1vmg krb5kdc[3518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 02:18:26 dist-test-slave-1vmg krb5kdc[3518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:26 dist-test-slave-1vmg krb5kdc[3518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 02:18:26.963970 3658 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:26.956122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46068 (local address 127.24.79.254:34655)
0808 02:18:26.956298 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:26.956303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:26.956443 (+ 140us) server_negotiation.cc:407] Connection header received
0808 02:18:26.956579 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:26.956583 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:26.956639 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:26.956756 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:26.957670 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.958467 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.959673 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.960041 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.960173 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.963821 (+ 3648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46068: BlockingRecv error: recv got EOF from 127.0.0.1:46068 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 08 02:18:26 dist-test-slave-1vmg krb5kdc[3518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM
I20260808 02:18:26.976368 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:26.965616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46072 (local address 127.24.79.254:34655)
0808 02:18:26.965755 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:26.965760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:26.965781 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:18:26.965935 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:26.965938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:26.965994 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:26.966100 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:26.966709 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.967623 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.968556 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:26.968951 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:26.969056 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:26.972216 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:26.972221 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:26.972224 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:26.972266 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:26.974127 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:26.974703 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:26.974707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:26.974709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:26.974829 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:26.975351 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:26.975355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:26.975356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:26.975597 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:26.975747 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:26.976037 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 02:18:26.976189 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 02:18:26.980261 3585 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46072:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:18:26.980921 3585 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:18:26.987164 3797 tablet_service.cc:1467] Processing CreateTablet for tablet d625d1f82f404a939e3aad88a3603bef (DEFAULT_TABLE table=test-table [id=cce117ab0b5248489324385b0bfeb67b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:26.987615 3797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d625d1f82f404a939e3aad88a3603bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:26.990386 4324 tablet_bootstrap.cc:492] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870: Bootstrap starting.
I20260808 02:18:26.991351 3997 tablet_service.cc:1467] Processing CreateTablet for tablet d625d1f82f404a939e3aad88a3603bef (DEFAULT_TABLE table=test-table [id=cce117ab0b5248489324385b0bfeb67b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:26.991629 4324 tablet_bootstrap.cc:654] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:26.991739 3997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d625d1f82f404a939e3aad88a3603bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:26.994495 4322 tablet_bootstrap.cc:492] T d625d1f82f404a939e3aad88a3603bef P df2742039bb84aa68ee5b5baefa8a161: Bootstrap starting.
I20260808 02:18:26.990839 4200 tablet_service.cc:1467] Processing CreateTablet for tablet d625d1f82f404a939e3aad88a3603bef (DEFAULT_TABLE table=test-table [id=cce117ab0b5248489324385b0bfeb67b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:26.995206 4200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d625d1f82f404a939e3aad88a3603bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:26.995620 4322 tablet_bootstrap.cc:654] T d625d1f82f404a939e3aad88a3603bef P df2742039bb84aa68ee5b5baefa8a161: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:26.998292 4325 tablet_bootstrap.cc:492] T d625d1f82f404a939e3aad88a3603bef P 5e877cad41a8409fa3f9bf84edfbbc79: Bootstrap starting.
I20260808 02:18:27.000026 4325 tablet_bootstrap.cc:654] T d625d1f82f404a939e3aad88a3603bef P 5e877cad41a8409fa3f9bf84edfbbc79: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:27.001510 4322 tablet_bootstrap.cc:492] T d625d1f82f404a939e3aad88a3603bef P df2742039bb84aa68ee5b5baefa8a161: No bootstrap required, opened a new log
I20260808 02:18:27.001641 4322 ts_tablet_manager.cc:1397] T d625d1f82f404a939e3aad88a3603bef P df2742039bb84aa68ee5b5baefa8a161: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260808 02:18:27.002193 4322 raft_consensus.cc:348] T d625d1f82f404a939e3aad88a3603bef P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:27.002336 4322 raft_consensus.cc:374] T d625d1f82f404a939e3aad88a3603bef P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:27.002367 4322 raft_consensus.cc:729] T d625d1f82f404a939e3aad88a3603bef P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df2742039bb84aa68ee5b5baefa8a161, State: Initialized, Role: FOLLOWER
I20260808 02:18:27.003383 4325 tablet_bootstrap.cc:492] T d625d1f82f404a939e3aad88a3603bef P 5e877cad41a8409fa3f9bf84edfbbc79: No bootstrap required, opened a new log
I20260808 02:18:27.003499 4325 ts_tablet_manager.cc:1397] T d625d1f82f404a939e3aad88a3603bef P 5e877cad41a8409fa3f9bf84edfbbc79: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 02:18:27.003996 4325 raft_consensus.cc:348] T d625d1f82f404a939e3aad88a3603bef P 5e877cad41a8409fa3f9bf84edfbbc79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:27.004113 4325 raft_consensus.cc:374] T d625d1f82f404a939e3aad88a3603bef P 5e877cad41a8409fa3f9bf84edfbbc79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:27.004144 4325 raft_consensus.cc:729] T d625d1f82f404a939e3aad88a3603bef P 5e877cad41a8409fa3f9bf84edfbbc79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e877cad41a8409fa3f9bf84edfbbc79, State: Initialized, Role: FOLLOWER
I20260808 02:18:27.005079 4324 tablet_bootstrap.cc:492] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870: No bootstrap required, opened a new log
I20260808 02:18:27.005195 4324 ts_tablet_manager.cc:1397] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20260808 02:18:27.005771 4324 raft_consensus.cc:348] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:27.005901 4324 raft_consensus.cc:374] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:27.005939 4324 raft_consensus.cc:729] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ca6b5f555cd4cfea50c6debdec39870, State: Initialized, Role: FOLLOWER
I20260808 02:18:27.006326 4322 consensus_queue.cc:260] T d625d1f82f404a939e3aad88a3603bef P df2742039bb84aa68ee5b5baefa8a161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:27.006702 4325 consensus_queue.cc:260] T d625d1f82f404a939e3aad88a3603bef P 5e877cad41a8409fa3f9bf84edfbbc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:27.007004 4324 consensus_queue.cc:260] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:27.010349 4339 raft_consensus.cc:482] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:18:27.010499 4339 raft_consensus.cc:504] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:27.012091 4339 leader_election.cc:288] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df2742039bb84aa68ee5b5baefa8a161 (127.24.79.194:46621), 5e877cad41a8409fa3f9bf84edfbbc79 (127.24.79.195:46391)
I20260808 02:18:27.014863 4325 ts_tablet_manager.cc:1428] T d625d1f82f404a939e3aad88a3603bef P 5e877cad41a8409fa3f9bf84edfbbc79: Time spent starting tablet: real 0.011s user 0.001s sys 0.000s
I20260808 02:18:27.015272 4324 ts_tablet_manager.cc:1428] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260808 02:18:27.015683 4322 ts_tablet_manager.cc:1428] T d625d1f82f404a939e3aad88a3603bef P df2742039bb84aa68ee5b5baefa8a161: Time spent starting tablet: real 0.014s user 0.000s sys 0.002s
I20260808 02:18:27.018985 4318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:27.013764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:36043 (local address 127.24.79.194:46621)
0808 02:18:27.014059 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:27.014068 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:27.014090 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:18:27.014167 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:27.014170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:27.014223 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:27.014312 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:27.014809 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:27.015587 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:27.017528 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:27.018106 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:27.018224 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:27.018306 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:27.018700 (+ 394us) server_negotiation.cc:299] Negotiation successful
0808 02:18:27.018813 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260808 02:18:27.019711 4424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:27.012454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:46621 (local address 127.24.79.193:36043)
0808 02:18:27.013365 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:27.013382 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:27.013518 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:27.014459 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:27.014463 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:27.014489 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:27.014652 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:27.014663 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:27.015942 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:27.015945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:27.017320 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:27.017381 (+ 61us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:27.018422 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:27.018425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:27.018535 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:27.018562 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 02:18:27.019441 (+ 879us) client_negotiation.cc:240] Negotiation successful
0808 02:18:27.019536 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"thread_start_us":206,"threads_started":1}
I20260808 02:18:27.020346 4025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d625d1f82f404a939e3aad88a3603bef" candidate_uuid: "7ca6b5f555cd4cfea50c6debdec39870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df2742039bb84aa68ee5b5baefa8a161" is_pre_election: true
I20260808 02:18:27.020663 4025 raft_consensus.cc:2454] T d625d1f82f404a939e3aad88a3603bef P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ca6b5f555cd4cfea50c6debdec39870 in term 0.
I20260808 02:18:27.021162 3721 leader_election.cc:302] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ca6b5f555cd4cfea50c6debdec39870, df2742039bb84aa68ee5b5baefa8a161; no voters:
I20260808 02:18:27.021476 4339 raft_consensus.cc:2781] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:18:27.021544 4339 raft_consensus.cc:482] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:18:27.021591 4339 raft_consensus.cc:3037] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:27.023113 4339 raft_consensus.cc:504] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:27.023702 4339 leader_election.cc:288] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870 [CANDIDATE]: Term 1 election: Requested vote from peers df2742039bb84aa68ee5b5baefa8a161 (127.24.79.194:46621), 5e877cad41a8409fa3f9bf84edfbbc79 (127.24.79.195:46391)
I20260808 02:18:27.024211 4423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:27.012823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:46391 (local address 127.24.79.193:60579)
0808 02:18:27.019514 (+ 6691us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:27.019530 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:27.019660 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:27.020010 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:27.020013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:27.020035 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:27.020191 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:27.020197 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:27.021311 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:27.021314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:27.022575 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:27.022588 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:27.023704 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:27.023707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:27.023817 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:27.023844 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 02:18:27.023955 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 02:18:27.024041 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6605,"thread_start_us":129,"threads_started":1}
I20260808 02:18:27.024192 4025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d625d1f82f404a939e3aad88a3603bef" candidate_uuid: "7ca6b5f555cd4cfea50c6debdec39870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df2742039bb84aa68ee5b5baefa8a161"
I20260808 02:18:27.024288 4321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:27.013564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:60579 (local address 127.24.79.195:46391)
0808 02:18:27.013732 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:27.013737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:27.019656 (+ 5919us) server_negotiation.cc:407] Connection header received
0808 02:18:27.019746 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:27.019749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:27.019811 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:27.019883 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:27.020340 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:27.021115 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:27.022750 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:27.023437 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:27.023538 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:27.023594 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:27.024012 (+ 418us) server_negotiation.cc:299] Negotiation successful
0808 02:18:27.024158 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:18:27.024333 4025 raft_consensus.cc:3037] T d625d1f82f404a939e3aad88a3603bef P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:27.025450 4025 raft_consensus.cc:2454] T d625d1f82f404a939e3aad88a3603bef P df2742039bb84aa68ee5b5baefa8a161 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ca6b5f555cd4cfea50c6debdec39870 in term 1.
I20260808 02:18:27.025857 3721 leader_election.cc:302] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ca6b5f555cd4cfea50c6debdec39870, df2742039bb84aa68ee5b5baefa8a161; no voters:
I20260808 02:18:27.026098 4339 raft_consensus.cc:2781] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:27.026290 4227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d625d1f82f404a939e3aad88a3603bef" candidate_uuid: "7ca6b5f555cd4cfea50c6debdec39870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e877cad41a8409fa3f9bf84edfbbc79"
I20260808 02:18:27.026424 4227 raft_consensus.cc:3037] T d625d1f82f404a939e3aad88a3603bef P 5e877cad41a8409fa3f9bf84edfbbc79 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:27.026466 4339 raft_consensus.cc:686] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870 [term 1 LEADER]: Becoming Leader. State: Replica: 7ca6b5f555cd4cfea50c6debdec39870, State: Running, Role: LEADER
I20260808 02:18:27.026849 4339 consensus_queue.cc:237] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:27.026254 4228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d625d1f82f404a939e3aad88a3603bef" candidate_uuid: "7ca6b5f555cd4cfea50c6debdec39870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" is_pre_election: true
I20260808 02:18:27.028905 4227 raft_consensus.cc:2454] T d625d1f82f404a939e3aad88a3603bef P 5e877cad41a8409fa3f9bf84edfbbc79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ca6b5f555cd4cfea50c6debdec39870 in term 1.
I20260808 02:18:27.029708 3585 catalog_manager.cc:5495] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ca6b5f555cd4cfea50c6debdec39870 (127.24.79.193). New cstate: current_term: 1 leader_uuid: "7ca6b5f555cd4cfea50c6debdec39870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } health_report { overall_health: UNKNOWN } } }
I20260808 02:18:27.042387 4320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:27.037965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57470 (local address 127.24.79.193:43433)
0808 02:18:27.038216 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:27.038222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:27.038247 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:18:27.038364 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:27.038367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:27.038446 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:27.038576 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:18:27.038969 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:27.039502 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:27.040669 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:27.040942 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:27.041037 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:27.041417 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:18:27.041964 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:18:27.042106 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:27.042136 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 02:18:27.042199 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260808 02:18:27.046294 4227 raft_consensus.cc:1261] T d625d1f82f404a939e3aad88a3603bef P 5e877cad41a8409fa3f9bf84edfbbc79 [term 1 FOLLOWER]: Refusing update from remote peer 7ca6b5f555cd4cfea50c6debdec39870: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:18:27.047164 4025 raft_consensus.cc:1261] T d625d1f82f404a939e3aad88a3603bef P df2742039bb84aa68ee5b5baefa8a161 [term 1 FOLLOWER]: Refusing update from remote peer 7ca6b5f555cd4cfea50c6debdec39870: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:18:27.047055 4339 consensus_queue.cc:1035] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:18:27.047876 4339 consensus_queue.cc:1035] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:18:27.063869 3585 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:18:27.064126 3585 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:18:27.067010 3585 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c: Sending DeleteTablet for 3 replicas of tablet d625d1f82f404a939e3aad88a3603bef
I20260808 02:18:27.067926 4200 tablet_service.cc:1514] Processing DeleteTablet for tablet d625d1f82f404a939e3aad88a3603bef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:27 UTC) from {username='oryx', principal='oryx/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:51318
I20260808 02:18:27.067927 3797 tablet_service.cc:1514] Processing DeleteTablet for tablet d625d1f82f404a939e3aad88a3603bef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:27 UTC) from {username='oryx', principal='oryx/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:57468
I20260808 02:18:27.068168 4361 tablet_replica.cc:330] T d625d1f82f404a939e3aad88a3603bef P 5e877cad41a8409fa3f9bf84edfbbc79: stopping tablet replica
I20260808 02:18:27.068276 4361 raft_consensus.cc:2229] T d625d1f82f404a939e3aad88a3603bef P 5e877cad41a8409fa3f9bf84edfbbc79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:18:27.068437 4361 raft_consensus.cc:2258] T d625d1f82f404a939e3aad88a3603bef P 5e877cad41a8409fa3f9bf84edfbbc79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:18:27.068678 4363 tablet_replica.cc:330] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870: stopping tablet replica
I20260808 02:18:27.068849 4363 raft_consensus.cc:2229] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:18:27.068863 3585 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46072:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 02:18:27.069283 4363 raft_consensus.cc:2258] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 02:18:27.069645 3585 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:18:27.069808 3997 tablet_service.cc:1514] Processing DeleteTablet for tablet d625d1f82f404a939e3aad88a3603bef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:27 UTC) from {username='oryx', principal='oryx/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:47988
I20260808 02:18:27.070039 4362 tablet_replica.cc:330] T d625d1f82f404a939e3aad88a3603bef P df2742039bb84aa68ee5b5baefa8a161: stopping tablet replica
I20260808 02:18:27.070168 4362 raft_consensus.cc:2229] T d625d1f82f404a939e3aad88a3603bef P df2742039bb84aa68ee5b5baefa8a161 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:18:27.070356 4362 raft_consensus.cc:2258] T d625d1f82f404a939e3aad88a3603bef P df2742039bb84aa68ee5b5baefa8a161 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:18:27.070525 4363 ts_tablet_manager.cc:1905] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:18:27.071436 4362 ts_tablet_manager.cc:1905] T d625d1f82f404a939e3aad88a3603bef P df2742039bb84aa68ee5b5baefa8a161: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:18:27.074100 4363 ts_tablet_manager.cc:1918] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:18:27.074198 4363 log.cc:1196] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-0/wal/wals/d625d1f82f404a939e3aad88a3603bef
I20260808 02:18:27.074421 4362 ts_tablet_manager.cc:1918] T d625d1f82f404a939e3aad88a3603bef P df2742039bb84aa68ee5b5baefa8a161: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:18:27.074455 4363 ts_tablet_manager.cc:1939] T d625d1f82f404a939e3aad88a3603bef P 7ca6b5f555cd4cfea50c6debdec39870: Deleting consensus metadata
I20260808 02:18:27.074506 4362 log.cc:1196] T d625d1f82f404a939e3aad88a3603bef P df2742039bb84aa68ee5b5baefa8a161: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/d625d1f82f404a939e3aad88a3603bef
I20260808 02:18:27.074870 4362 ts_tablet_manager.cc:1939] T d625d1f82f404a939e3aad88a3603bef P df2742039bb84aa68ee5b5baefa8a161: Deleting consensus metadata
I20260808 02:18:27.075292 4361 ts_tablet_manager.cc:1905] T d625d1f82f404a939e3aad88a3603bef P 5e877cad41a8409fa3f9bf84edfbbc79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:18:27.076856 3797 tablet_service.cc:1467] Processing CreateTablet for tablet a1d84859be1841a28f268815e3d2b9e5 (DEFAULT_TABLE table=test-table [id=cfec55d3ab714b90bb0d69d94b866ec8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:27.077212 3797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1d84859be1841a28f268815e3d2b9e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:27.078233 4361 ts_tablet_manager.cc:1918] T d625d1f82f404a939e3aad88a3603bef P 5e877cad41a8409fa3f9bf84edfbbc79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:18:27.078344 4361 log.cc:1196] T d625d1f82f404a939e3aad88a3603bef P 5e877cad41a8409fa3f9bf84edfbbc79: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/d625d1f82f404a939e3aad88a3603bef
I20260808 02:18:27.078641 4361 ts_tablet_manager.cc:1939] T d625d1f82f404a939e3aad88a3603bef P 5e877cad41a8409fa3f9bf84edfbbc79: Deleting consensus metadata
I20260808 02:18:27.079113 3565 catalog_manager.cc:4841] TS 7ca6b5f555cd4cfea50c6debdec39870 (127.24.79.193:43433): tablet d625d1f82f404a939e3aad88a3603bef (table test-table [id=cce117ab0b5248489324385b0bfeb67b]) successfully deleted
I20260808 02:18:27.079422 4200 tablet_service.cc:1467] Processing CreateTablet for tablet a1d84859be1841a28f268815e3d2b9e5 (DEFAULT_TABLE table=test-table [id=cfec55d3ab714b90bb0d69d94b866ec8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:27.079751 4200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1d84859be1841a28f268815e3d2b9e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:27.079924 4324 tablet_bootstrap.cc:492] T a1d84859be1841a28f268815e3d2b9e5 P 7ca6b5f555cd4cfea50c6debdec39870: Bootstrap starting.
I20260808 02:18:27.081027 4324 tablet_bootstrap.cc:654] T a1d84859be1841a28f268815e3d2b9e5 P 7ca6b5f555cd4cfea50c6debdec39870: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:27.082556 4325 tablet_bootstrap.cc:492] T a1d84859be1841a28f268815e3d2b9e5 P 5e877cad41a8409fa3f9bf84edfbbc79: Bootstrap starting.
I20260808 02:18:27.083141 3997 tablet_service.cc:1467] Processing CreateTablet for tablet a1d84859be1841a28f268815e3d2b9e5 (DEFAULT_TABLE table=test-table [id=cfec55d3ab714b90bb0d69d94b866ec8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:27.083473 3997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1d84859be1841a28f268815e3d2b9e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:27.083688 4325 tablet_bootstrap.cc:654] T a1d84859be1841a28f268815e3d2b9e5 P 5e877cad41a8409fa3f9bf84edfbbc79: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:27.083782 4324 tablet_bootstrap.cc:492] T a1d84859be1841a28f268815e3d2b9e5 P 7ca6b5f555cd4cfea50c6debdec39870: No bootstrap required, opened a new log
I20260808 02:18:27.083884 4324 ts_tablet_manager.cc:1397] T a1d84859be1841a28f268815e3d2b9e5 P 7ca6b5f555cd4cfea50c6debdec39870: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 02:18:27.084422 4324 raft_consensus.cc:348] T a1d84859be1841a28f268815e3d2b9e5 P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:27.084568 4324 raft_consensus.cc:374] T a1d84859be1841a28f268815e3d2b9e5 P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:27.084612 4324 raft_consensus.cc:729] T a1d84859be1841a28f268815e3d2b9e5 P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ca6b5f555cd4cfea50c6debdec39870, State: Initialized, Role: FOLLOWER
I20260808 02:18:27.084774 4324 consensus_queue.cc:260] T a1d84859be1841a28f268815e3d2b9e5 P 7ca6b5f555cd4cfea50c6debdec39870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:27.085111 4324 ts_tablet_manager.cc:1428] T a1d84859be1841a28f268815e3d2b9e5 P 7ca6b5f555cd4cfea50c6debdec39870: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 02:18:27.086248 4325 tablet_bootstrap.cc:492] T a1d84859be1841a28f268815e3d2b9e5 P 5e877cad41a8409fa3f9bf84edfbbc79: No bootstrap required, opened a new log
I20260808 02:18:27.086364 4325 ts_tablet_manager.cc:1397] T a1d84859be1841a28f268815e3d2b9e5 P 5e877cad41a8409fa3f9bf84edfbbc79: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260808 02:18:27.086527 4322 tablet_bootstrap.cc:492] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161: Bootstrap starting.
I20260808 02:18:27.086877 4325 raft_consensus.cc:348] T a1d84859be1841a28f268815e3d2b9e5 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:27.087037 4325 raft_consensus.cc:374] T a1d84859be1841a28f268815e3d2b9e5 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:27.087070 4325 raft_consensus.cc:729] T a1d84859be1841a28f268815e3d2b9e5 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e877cad41a8409fa3f9bf84edfbbc79, State: Initialized, Role: FOLLOWER
I20260808 02:18:27.087206 4325 consensus_queue.cc:260] T a1d84859be1841a28f268815e3d2b9e5 P 5e877cad41a8409fa3f9bf84edfbbc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:27.087463 4325 ts_tablet_manager.cc:1428] T a1d84859be1841a28f268815e3d2b9e5 P 5e877cad41a8409fa3f9bf84edfbbc79: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:18:27.087541 4322 tablet_bootstrap.cc:654] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:27.089031 3565 catalog_manager.cc:4841] TS 5e877cad41a8409fa3f9bf84edfbbc79 (127.24.79.195:46391): tablet d625d1f82f404a939e3aad88a3603bef (table test-table [id=cce117ab0b5248489324385b0bfeb67b]) successfully deleted
I20260808 02:18:27.089823 3562 catalog_manager.cc:4841] TS df2742039bb84aa68ee5b5baefa8a161 (127.24.79.194:46621): tablet d625d1f82f404a939e3aad88a3603bef (table test-table [id=cce117ab0b5248489324385b0bfeb67b]) successfully deleted
I20260808 02:18:27.090814 4322 tablet_bootstrap.cc:492] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161: No bootstrap required, opened a new log
I20260808 02:18:27.091003 4322 ts_tablet_manager.cc:1397] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 02:18:27.091528 4322 raft_consensus.cc:348] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:27.091681 4322 raft_consensus.cc:374] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:27.091719 4322 raft_consensus.cc:729] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df2742039bb84aa68ee5b5baefa8a161, State: Initialized, Role: FOLLOWER
I20260808 02:18:27.091864 4322 consensus_queue.cc:260] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:27.092267 4322 ts_tablet_manager.cc:1428] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:18:27.112202 4341 raft_consensus.cc:482] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:18:27.112483 4341 raft_consensus.cc:504] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:27.113027 4341 leader_election.cc:288] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ca6b5f555cd4cfea50c6debdec39870 (127.24.79.193:43433), 5e877cad41a8409fa3f9bf84edfbbc79 (127.24.79.195:46391)
I20260808 02:18:27.113726 3827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1d84859be1841a28f268815e3d2b9e5" candidate_uuid: "df2742039bb84aa68ee5b5baefa8a161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca6b5f555cd4cfea50c6debdec39870" is_pre_election: true
I20260808 02:18:27.113721 4227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1d84859be1841a28f268815e3d2b9e5" candidate_uuid: "df2742039bb84aa68ee5b5baefa8a161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" is_pre_election: true
I20260808 02:18:27.113900 4227 raft_consensus.cc:2454] T a1d84859be1841a28f268815e3d2b9e5 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df2742039bb84aa68ee5b5baefa8a161 in term 0.
I20260808 02:18:27.113903 3827 raft_consensus.cc:2454] T a1d84859be1841a28f268815e3d2b9e5 P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df2742039bb84aa68ee5b5baefa8a161 in term 0.
I20260808 02:18:27.114318 3931 leader_election.cc:302] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ca6b5f555cd4cfea50c6debdec39870, df2742039bb84aa68ee5b5baefa8a161; no voters:
I20260808 02:18:27.114570 4341 raft_consensus.cc:2781] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:18:27.114655 4341 raft_consensus.cc:482] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:18:27.114694 4341 raft_consensus.cc:3037] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:27.116267 4341 raft_consensus.cc:504] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:27.116756 4341 leader_election.cc:288] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161 [CANDIDATE]: Term 1 election: Requested vote from peers 7ca6b5f555cd4cfea50c6debdec39870 (127.24.79.193:43433), 5e877cad41a8409fa3f9bf84edfbbc79 (127.24.79.195:46391)
I20260808 02:18:27.117398 3827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1d84859be1841a28f268815e3d2b9e5" candidate_uuid: "df2742039bb84aa68ee5b5baefa8a161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca6b5f555cd4cfea50c6debdec39870"
I20260808 02:18:27.117571 3827 raft_consensus.cc:3037] T a1d84859be1841a28f268815e3d2b9e5 P 7ca6b5f555cd4cfea50c6debdec39870 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:27.119182 4227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1d84859be1841a28f268815e3d2b9e5" candidate_uuid: "df2742039bb84aa68ee5b5baefa8a161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e877cad41a8409fa3f9bf84edfbbc79"
I20260808 02:18:27.119349 4227 raft_consensus.cc:3037] T a1d84859be1841a28f268815e3d2b9e5 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:27.119403 3827 raft_consensus.cc:2454] T a1d84859be1841a28f268815e3d2b9e5 P 7ca6b5f555cd4cfea50c6debdec39870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df2742039bb84aa68ee5b5baefa8a161 in term 1.
I20260808 02:18:27.119915 3931 leader_election.cc:302] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ca6b5f555cd4cfea50c6debdec39870, df2742039bb84aa68ee5b5baefa8a161; no voters:
I20260808 02:18:27.120200 4341 raft_consensus.cc:2781] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:27.120321 4341 raft_consensus.cc:686] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161 [term 1 LEADER]: Becoming Leader. State: Replica: df2742039bb84aa68ee5b5baefa8a161, State: Running, Role: LEADER
I20260808 02:18:27.120489 4341 consensus_queue.cc:237] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } }
I20260808 02:18:27.120806 4227 raft_consensus.cc:2454] T a1d84859be1841a28f268815e3d2b9e5 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df2742039bb84aa68ee5b5baefa8a161 in term 1.
I20260808 02:18:27.122658 3586 catalog_manager.cc:5495] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161 reported cstate change: term changed from 0 to 1, leader changed from <none> to df2742039bb84aa68ee5b5baefa8a161 (127.24.79.194). New cstate: current_term: 1 leader_uuid: "df2742039bb84aa68ee5b5baefa8a161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df2742039bb84aa68ee5b5baefa8a161" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 46621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 } health_report { overall_health: UNKNOWN } } }
I20260808 02:18:27.149305 4318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:27.144674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48016 (local address 127.24.79.194:46621)
0808 02:18:27.144926 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:27.144931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:27.144975 (+ 44us) server_negotiation.cc:407] Connection header received
0808 02:18:27.145025 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:27.145027 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:27.145313 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:27.145398 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:18:27.145757 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:27.146361 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:27.147744 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:27.147997 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:27.148104 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:27.148685 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:18:27.148792 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:18:27.148868 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:27.149041 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 02:18:27.149115 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 02:18:27.157382 4227 raft_consensus.cc:1261] T a1d84859be1841a28f268815e3d2b9e5 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 1 FOLLOWER]: Refusing update from remote peer df2742039bb84aa68ee5b5baefa8a161: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:18:27.157599 3828 raft_consensus.cc:1261] T a1d84859be1841a28f268815e3d2b9e5 P 7ca6b5f555cd4cfea50c6debdec39870 [term 1 FOLLOWER]: Refusing update from remote peer df2742039bb84aa68ee5b5baefa8a161: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:18:27.158171 4356 consensus_queue.cc:1035] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e877cad41a8409fa3f9bf84edfbbc79" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 46391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:18:27.158524 4341 consensus_queue.cc:1035] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ca6b5f555cd4cfea50c6debdec39870" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 43433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:18:27.204406 3585 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:18:27.204631 3585 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:18:27.207715 3585 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d485b7e961b4b41acdb265ce146848c: Sending DeleteTablet for 3 replicas of tablet a1d84859be1841a28f268815e3d2b9e5
I20260808 02:18:27.208556 4200 tablet_service.cc:1514] Processing DeleteTablet for tablet a1d84859be1841a28f268815e3d2b9e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:27 UTC) from {username='oryx', principal='oryx/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:51318
I20260808 02:18:27.208541 3997 tablet_service.cc:1514] Processing DeleteTablet for tablet a1d84859be1841a28f268815e3d2b9e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:27 UTC) from {username='oryx', principal='oryx/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:47988
I20260808 02:18:27.208778 4361 tablet_replica.cc:330] T a1d84859be1841a28f268815e3d2b9e5 P 5e877cad41a8409fa3f9bf84edfbbc79: stopping tablet replica
I20260808 02:18:27.208886 4361 raft_consensus.cc:2229] T a1d84859be1841a28f268815e3d2b9e5 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:18:27.209015 4361 raft_consensus.cc:2258] T a1d84859be1841a28f268815e3d2b9e5 P 5e877cad41a8409fa3f9bf84edfbbc79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:18:27.209204 4362 tablet_replica.cc:330] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161: stopping tablet replica
I20260808 02:18:27.209333 4362 raft_consensus.cc:2229] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:18:27.209406 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 3692
I20260808 02:18:27.209583 4362 raft_consensus.cc:2258] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:18:27.222087 4361 ts_tablet_manager.cc:1905] T a1d84859be1841a28f268815e3d2b9e5 P 5e877cad41a8409fa3f9bf84edfbbc79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:18:27.225449 4361 ts_tablet_manager.cc:1918] T a1d84859be1841a28f268815e3d2b9e5 P 5e877cad41a8409fa3f9bf84edfbbc79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:18:27.225575 4361 log.cc:1196] T a1d84859be1841a28f268815e3d2b9e5 P 5e877cad41a8409fa3f9bf84edfbbc79: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/a1d84859be1841a28f268815e3d2b9e5
I20260808 02:18:27.225821 4361 ts_tablet_manager.cc:1939] T a1d84859be1841a28f268815e3d2b9e5 P 5e877cad41a8409fa3f9bf84edfbbc79: Deleting consensus metadata
I20260808 02:18:27.227383 3565 catalog_manager.cc:4841] TS 5e877cad41a8409fa3f9bf84edfbbc79 (127.24.79.195:46391): tablet a1d84859be1841a28f268815e3d2b9e5 (table test-table [id=cfec55d3ab714b90bb0d69d94b866ec8]) successfully deleted
I20260808 02:18:27.227909 4362 ts_tablet_manager.cc:1905] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:18:27.230399 4362 ts_tablet_manager.cc:1918] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:18:27.230505 4362 log.cc:1196] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/a1d84859be1841a28f268815e3d2b9e5
I20260808 02:18:27.230731 4362 ts_tablet_manager.cc:1939] T a1d84859be1841a28f268815e3d2b9e5 P df2742039bb84aa68ee5b5baefa8a161: Deleting consensus metadata
I20260808 02:18:27.231793 3562 catalog_manager.cc:4841] TS df2742039bb84aa68ee5b5baefa8a161 (127.24.79.194:46621): tablet a1d84859be1841a28f268815e3d2b9e5 (table test-table [id=cfec55d3ab714b90bb0d69d94b866ec8]) successfully deleted
W20260808 02:18:27.235515 3565 connection.cc:535] client connection to 127.24.79.193:43433 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 02:18:27.235707 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 3909
W20260808 02:18:27.235764 3565 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 02:18:27.247606 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:27.245346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:43433 (local address 127.0.0.1:57486)
0808 02:18:27.247350 (+ 2004us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:27.247477 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.193:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1900,"thread_start_us":322,"threads_started":1}
W20260808 02:18:27.247843 3565 catalog_manager.cc:4584] TS 7ca6b5f555cd4cfea50c6debdec39870 (127.24.79.193:43433): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a1d84859be1841a28f268815e3d2b9e5: Network error: Client connection negotiation failed: client connection to 127.24.79.193:43433: connect: Connection refused (error 111)
I20260808 02:18:27.255337 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 4106
I20260808 02:18:27.279062 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 3539
2026-08-08T02:18:27Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3169 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:18:27 dist-test-slave-1vmg krb5kdc[4449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:18:27 dist-test-slave-1vmg krb5kdc[4449](info): set up 2 sockets
Aug 08 02:18:27 dist-test-slave-1vmg krb5kdc[4449](info): commencing operation
krb5kdc: starting...
W20260808 02:18:28.458084 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:18:28 dist-test-slave-1vmg krb5kdc[4449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:18:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:18:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:18:28.612232 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:36813
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:40403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:36813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:28.741457 4467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:28.741804 4467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:28.741854 4467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:28.746275 4467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:18:28.746389 4467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:28.746409 4467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:28.746425 4467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:18:28.746443 4467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:18:28.752383 4467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:36813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:36813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:28.753716 4467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:28.754989 4467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:28.766777 4475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:28.766791 4474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:28.767117 4467 server_base.cc:1034] running on GCE node
W20260808 02:18:28.766780 4477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:28.767999 4467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:28.769445 4467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:28.770751 4467 hybrid_clock.cc:648] HybridClock initialized: now 1786155508770673 us; error 96 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T02:18:28Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1491 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:18:28 dist-test-slave-1vmg krb5kdc[4482](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:18:28 dist-test-slave-1vmg krb5kdc[4482](info): set up 2 sockets
Aug 08 02:18:28 dist-test-slave-1vmg krb5kdc[4482](info): commencing operation
krb5kdc: starting...
W20260808 02:18:29.966885 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:18:30 dist-test-slave-1vmg krb5kdc[4482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:18:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:18:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:18:30.131235 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:35845
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:35845
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:30.262862 4499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:30.263307 4499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:30.263373 4499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:30.267697 4499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:18:30.267799 4499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:30.267829 4499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:30.267851 4499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:18:30.267874 4499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:18:30.274340 4499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:35845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:35845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:30.275789 4499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:30.277077 4499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:30.287343 4509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:30.287344 4506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:30.287382 4507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:30.287606 4499 server_base.cc:1034] running on GCE node
I20260808 02:18:30.288470 4499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:30.289872 4499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:30.291234 4499 hybrid_clock.cc:648] HybridClock initialized: now 1786155510291192 us; error 61 us; skew 500 ppm
Aug 08 02:18:30 dist-test-slave-1vmg krb5kdc[4482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155510, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:30.299486 4499 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:18:30.300781 4499 webserver.cc:460] Webserver started at http://127.24.79.254:43929/ using document root <none> and password file <none>
I20260808 02:18:30.301481 4499 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:30.301566 4499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:30.301826 4499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:30.304402 4499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "6d77fc9637e1432aa762f74aefba1843"
format_stamp: "Formatted at 2026-08-08 02:18:30 on dist-test-slave-1vmg"
server_key: "3b62c4a382ac30aa4cdcc1583604d8fc"
server_key_iv: "b21d061e5dbd1c8176c1b10ea7a36867"
server_key_version: "encryptionkey@0"
I20260808 02:18:30.305056 4499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "6d77fc9637e1432aa762f74aefba1843"
format_stamp: "Formatted at 2026-08-08 02:18:30 on dist-test-slave-1vmg"
server_key: "3b62c4a382ac30aa4cdcc1583604d8fc"
server_key_iv: "b21d061e5dbd1c8176c1b10ea7a36867"
server_key_version: "encryptionkey@0"
I20260808 02:18:30.309664 4499 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:18:30.312883 4520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:30.314599 4499 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 02:18:30.314767 4499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "6d77fc9637e1432aa762f74aefba1843"
format_stamp: "Formatted at 2026-08-08 02:18:30 on dist-test-slave-1vmg"
server_key: "3b62c4a382ac30aa4cdcc1583604d8fc"
server_key_iv: "b21d061e5dbd1c8176c1b10ea7a36867"
server_key_version: "encryptionkey@0"
I20260808 02:18:30.314905 4499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:30.325608 4499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:30.329376 4499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:30.329602 4499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:30.338188 4499 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:35845
I20260808 02:18:30.338197 4599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:35845 every 8 connection(s)
I20260808 02:18:30.339411 4499 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:18:30.341466 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 4499
I20260808 02:18:30.341626 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:18:30.342105 24895 external_mini_cluster.cc:1428] Setting key 1148ee89a8861a8066f6eb721c2ef2d6
I20260808 02:18:30.343537 4600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:30.351781 4600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843: Bootstrap starting.
Aug 08 02:18:30 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:18:30.355090 4600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:30.356040 4600 log.cc:824] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:30.358877 4600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843: No bootstrap required, opened a new log
I20260808 02:18:30.360455 4605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:30.343933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49184 (local address 127.24.79.254:35845)
0808 02:18:30.344518 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:30.344538 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:30.344582 (+ 44us) server_negotiation.cc:407] Connection header received
0808 02:18:30.345339 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:30.345362 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:30.345782 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:30.346135 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:30.347496 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:30.348633 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:30.349971 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:30.350632 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:30.350791 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:30.354283 (+ 3492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:30.354290 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:30.354308 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:30.354351 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:30.356405 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:30.357020 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:30.357027 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:30.357103 (+ 76us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:30.357289 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:30.357747 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:30.357751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:30.357752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:30.358313 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:30.358503 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:30.359001 (+ 498us) server_negotiation.cc:299] Negotiation successful
0808 02:18:30.359414 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":184,"threads_started":1}
I20260808 02:18:30.363204 4600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d77fc9637e1432aa762f74aefba1843" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 35845 } }
I20260808 02:18:30.363529 4600 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:30.363577 4600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d77fc9637e1432aa762f74aefba1843, State: Initialized, Role: FOLLOWER
I20260808 02:18:30.364058 4600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d77fc9637e1432aa762f74aefba1843" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 35845 } }
I20260808 02:18:30.364212 4600 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:18:30.364259 4600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:18:30.364367 4600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:30.365487 4600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d77fc9637e1432aa762f74aefba1843" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 35845 } }
I20260808 02:18:30.365885 4600 leader_election.cc:302] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d77fc9637e1432aa762f74aefba1843; no voters:
I20260808 02:18:30.366254 4600 leader_election.cc:288] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:18:30.366482 4607 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:30.366827 4607 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [term 1 LEADER]: Becoming Leader. State: Replica: 6d77fc9637e1432aa762f74aefba1843, State: Running, Role: LEADER
I20260808 02:18:30.367375 4607 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d77fc9637e1432aa762f74aefba1843" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 35845 } }
I20260808 02:18:30.367823 4600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:18:30.369725 4608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d77fc9637e1432aa762f74aefba1843" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d77fc9637e1432aa762f74aefba1843" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 35845 } } }
I20260808 02:18:30.369923 4608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:30.370471 4609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d77fc9637e1432aa762f74aefba1843. Latest consensus state: current_term: 1 leader_uuid: "6d77fc9637e1432aa762f74aefba1843" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d77fc9637e1432aa762f74aefba1843" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 35845 } } }
I20260808 02:18:30.371062 4609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [sys.catalog]: This master's current role is: LEADER
W20260808 02:18:30.374531 4626 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:18:30.374737 4626 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:18:30.374835 4624 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:18:30.375800 4624 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:18:30.382458 4624 catalog_manager.cc:1347] Generated new cluster ID: d8b529e010a342e09e30abbe761a392c
I20260808 02:18:30.382571 4624 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:18:30.410684 4624 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:18:30.412245 4624 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:18:30.427989 4624 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843: Generated new TSK 0
I20260808 02:18:30.428890 4624 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.79.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
I20260808 02:18:30.472748 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:35845
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:30.606452 4633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:30.606807 4633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:30.606874 4633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:30.611263 4633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:30.611349 4633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:30.611436 4633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:18:30.617532 4633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:35845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:30.618899 4633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:30.620258 4633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:30.631692 4645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:30.631726 4642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:30.632059 4633 server_base.cc:1034] running on GCE node
W20260808 02:18:30.631738 4641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:30.632790 4633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:30.633558 4633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:30.634779 4633 hybrid_clock.cc:648] HybridClock initialized: now 1786155510634753 us; error 40 us; skew 500 ppm
Aug 08 02:18:30 dist-test-slave-1vmg krb5kdc[4482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155510, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:30.638091 4633 init.cc:377] Logged in from keytab as kudu/127.24.79.193@KRBTEST.COM (short username kudu)
I20260808 02:18:30.639283 4633 webserver.cc:460] Webserver started at http://127.24.79.193:40723/ using document root <none> and password file <none>
I20260808 02:18:30.639950 4633 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:30.640029 4633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:30.640280 4633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:30.642597 4633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "e8d933c8490743af89d331556c803f83"
format_stamp: "Formatted at 2026-08-08 02:18:30 on dist-test-slave-1vmg"
server_key: "60e95f500eb6d69dd1781d13ba793159"
server_key_iv: "51b94f111f96b5c666873b934cf5eda5"
server_key_version: "encryptionkey@0"
I20260808 02:18:30.643271 4633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "e8d933c8490743af89d331556c803f83"
format_stamp: "Formatted at 2026-08-08 02:18:30 on dist-test-slave-1vmg"
server_key: "60e95f500eb6d69dd1781d13ba793159"
server_key_iv: "51b94f111f96b5c666873b934cf5eda5"
server_key_version: "encryptionkey@0"
I20260808 02:18:30.647974 4633 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:18:30.651678 4652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:30.653338 4633 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:18:30.653501 4633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "e8d933c8490743af89d331556c803f83"
format_stamp: "Formatted at 2026-08-08 02:18:30 on dist-test-slave-1vmg"
server_key: "60e95f500eb6d69dd1781d13ba793159"
server_key_iv: "51b94f111f96b5c666873b934cf5eda5"
server_key_version: "encryptionkey@0"
I20260808 02:18:30.653648 4633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:30.684767 4633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:30.688560 4633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:30.688783 4633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:30.689473 4633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:30.690419 4633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:30.690470 4633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:30.690531 4633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:30.690555 4633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:30.700412 4633 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:36337
I20260808 02:18:30.700512 4826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:36337 every 8 connection(s)
I20260808 02:18:30.701490 4633 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:18:30.703145 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 4633
I20260808 02:18:30.703310 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:18:30.703715 24895 external_mini_cluster.cc:1428] Setting key 4ac3757a249cfcb7fb52373990531b73
Aug 08 02:18:30 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155510, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.79.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.194@KRBTEST.COM" created.
I20260808 02:18:30.718650 4605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:30.703857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:40511 (local address 127.24.79.254:35845)
0808 02:18:30.704001 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:30.704005 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:30.705150 (+ 1145us) server_negotiation.cc:407] Connection header received
0808 02:18:30.706693 (+ 1543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:30.706697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:30.706761 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:30.706858 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:30.708419 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:30.708976 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:30.710295 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:30.710564 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:30.710666 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:30.713854 (+ 3188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:30.713859 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:30.713862 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:30.713898 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:30.715814 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:30.716500 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:30.716504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:30.716505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:30.716606 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:30.717066 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:30.717070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:30.717071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:30.717361 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:30.717457 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:30.718256 (+ 799us) server_negotiation.cc:299] Negotiation successful
0808 02:18:30.718433 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 02:18:30.719868 4832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:30.704374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.193:40511)
0808 02:18:30.704979 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:30.705031 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:30.706405 (+ 1374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:30.707114 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:30.707123 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:30.707566 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:30.708234 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:30.708252 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:30.709130 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:30.709134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:30.710119 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:30.710133 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:30.710760 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:30.710765 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:30.711032 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:30.711706 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:30.711734 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:30.713535 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:30.715987 (+ 2452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:30.715996 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:30.716015 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:30.716363 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:30.716721 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:30.716723 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:30.716725 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:30.716905 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:30.717509 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:30.717517 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:30.717892 (+ 375us) client_negotiation.cc:769] Sending connection context
0808 02:18:30.718121 (+ 229us) client_negotiation.cc:240] Negotiation successful
0808 02:18:30.718544 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":162,"threads_started":1}
I20260808 02:18:30.721577 4829 heartbeater.cc:347] Connected to a master server at 127.24.79.254:35845
I20260808 02:18:30.722014 4829 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:30.722821 4829 heartbeater.cc:511] Master 127.24.79.254:35845 requested a full tablet report, sending...
I20260808 02:18:30.725159 4545 ts_manager.cc:194] Registered new tserver with Master: e8d933c8490743af89d331556c803f83 (127.24.79.193:36337)
I20260808 02:18:30.726521 4545 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.193@KRBTEST.COM'} at 127.24.79.193:40511
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
I20260808 02:18:30.743634 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:35845
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:30.876435 4837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:30.876783 4837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:30.876832 4837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:30.881230 4837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:30.881337 4837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:30.881404 4837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:18:30.887442 4837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:35845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:30.888688 4837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:30.889844 4837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:30.901183 4844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:30.901283 4837 server_base.cc:1034] running on GCE node
W20260808 02:18:30.901189 4843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:30.901183 4846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:30.901990 4837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:30.902773 4837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:30.903966 4837 hybrid_clock.cc:648] HybridClock initialized: now 1786155510903886 us; error 92 us; skew 500 ppm
Aug 08 02:18:30 dist-test-slave-1vmg krb5kdc[4482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155510, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:30.907286 4837 init.cc:377] Logged in from keytab as kudu/127.24.79.194@KRBTEST.COM (short username kudu)
I20260808 02:18:30.908406 4837 webserver.cc:460] Webserver started at http://127.24.79.194:43543/ using document root <none> and password file <none>
I20260808 02:18:30.909034 4837 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:30.909102 4837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:30.909328 4837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:30.911895 4837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "857f3353aec1432d8244a67f557a599b"
format_stamp: "Formatted at 2026-08-08 02:18:30 on dist-test-slave-1vmg"
server_key: "48a54f6ef10c8b862a3a49bcc224a109"
server_key_iv: "3d856de6574f01897f8475e5aa38a6e3"
server_key_version: "encryptionkey@0"
I20260808 02:18:30.912550 4837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "857f3353aec1432d8244a67f557a599b"
format_stamp: "Formatted at 2026-08-08 02:18:30 on dist-test-slave-1vmg"
server_key: "48a54f6ef10c8b862a3a49bcc224a109"
server_key_iv: "3d856de6574f01897f8475e5aa38a6e3"
server_key_version: "encryptionkey@0"
I20260808 02:18:30.916926 4837 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 02:18:30.920056 4853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:30.921720 4837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:18:30.921859 4837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "857f3353aec1432d8244a67f557a599b"
format_stamp: "Formatted at 2026-08-08 02:18:30 on dist-test-slave-1vmg"
server_key: "48a54f6ef10c8b862a3a49bcc224a109"
server_key_iv: "3d856de6574f01897f8475e5aa38a6e3"
server_key_version: "encryptionkey@0"
I20260808 02:18:30.921980 4837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:30.949397 4837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:30.953452 4837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:30.953646 4837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:30.954301 4837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:30.955343 4837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:30.955401 4837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:30.955447 4837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:30.955468 4837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:30.966082 4837 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:42051
I20260808 02:18:30.966120 5009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:42051 every 8 connection(s)
I20260808 02:18:30.967231 4837 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:18:30.973573 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 4837
I20260808 02:18:30.973701 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:18:30.973989 24895 external_mini_cluster.cc:1428] Setting key 628f6544db26a1ac00106396e80e8b23
Aug 08 02:18:30 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155510, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:18:30.984375 4605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:30.969525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:54845 (local address 127.24.79.254:35845)
0808 02:18:30.969665 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:30.969669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:30.970770 (+ 1101us) server_negotiation.cc:407] Connection header received
0808 02:18:30.972369 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:30.972373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:30.972444 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:30.972527 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:30.974034 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:30.974724 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:30.976067 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:30.976311 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:30.976405 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:30.979684 (+ 3279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:30.979689 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:30.979692 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:30.979732 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:30.981492 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:30.982225 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:30.982230 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:30.982232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:30.982342 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:30.982834 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:30.982839 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:30.982841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:30.983175 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:30.983302 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:30.983993 (+ 691us) server_negotiation.cc:299] Negotiation successful
0808 02:18:30.984172 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 02:18:30.985663 5013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:30.970016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.194:54845)
0808 02:18:30.970562 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:30.970613 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:30.972075 (+ 1462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:30.972762 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:30.972771 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:30.973219 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:30.973843 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:30.973863 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:30.974867 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:30.974870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:30.975925 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:30.975938 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:30.976481 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:30.976487 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:30.976682 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:30.977575 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:30.977596 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:30.979386 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:30.981714 (+ 2328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:30.981721 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:30.981742 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:30.982086 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:30.982474 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:30.982477 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:30.982478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:30.982698 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:30.983319 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:30.983327 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:30.983689 (+ 362us) client_negotiation.cc:769] Sending connection context
0808 02:18:30.983886 (+ 197us) client_negotiation.cc:240] Negotiation successful
0808 02:18:30.984347 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":150,"threads_started":1}
WARNING: no policy specified for kudu/127.24.79.195@KRBTEST.COM; defaulting to no policy
I20260808 02:18:30.987341 5010 heartbeater.cc:347] Connected to a master server at 127.24.79.254:35845
I20260808 02:18:30.987673 5010 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:30.988294 5010 heartbeater.cc:511] Master 127.24.79.254:35845 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.195@KRBTEST.COM" created.
I20260808 02:18:30.990046 4545 ts_manager.cc:194] Registered new tserver with Master: 857f3353aec1432d8244a67f557a599b (127.24.79.194:42051)
I20260808 02:18:30.990824 4545 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.194@KRBTEST.COM'} at 127.24.79.194:54845
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
I20260808 02:18:31.011384 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:35845
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:31.138846 5016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:31.139302 5016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:31.139359 5016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:31.143899 5016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:31.143996 5016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:31.144063 5016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:18:31.150115 5016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:35845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:31.151535 5016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:31.152729 5016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:31.164729 5029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:31.164973 5027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:31.164999 5026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:31.165372 5016 server_base.cc:1034] running on GCE node
I20260808 02:18:31.165932 5016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:31.166718 5016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:31.167934 5016 hybrid_clock.cc:648] HybridClock initialized: now 1786155511167896 us; error 60 us; skew 500 ppm
Aug 08 02:18:31 dist-test-slave-1vmg krb5kdc[4482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155511, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:31.171944 5016 init.cc:377] Logged in from keytab as kudu/127.24.79.195@KRBTEST.COM (short username kudu)
I20260808 02:18:31.173132 5016 webserver.cc:460] Webserver started at http://127.24.79.195:40065/ using document root <none> and password file <none>
I20260808 02:18:31.173895 5016 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:31.173983 5016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:31.174203 5016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:31.176661 5016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "4aa9240bda48431ea69e687e2587efe8"
format_stamp: "Formatted at 2026-08-08 02:18:31 on dist-test-slave-1vmg"
server_key: "fd774ae300f53a5ccfb62f365d628acf"
server_key_iv: "30a6ec5b8a390316d9d820795bee64b9"
server_key_version: "encryptionkey@0"
I20260808 02:18:31.177357 5016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "4aa9240bda48431ea69e687e2587efe8"
format_stamp: "Formatted at 2026-08-08 02:18:31 on dist-test-slave-1vmg"
server_key: "fd774ae300f53a5ccfb62f365d628acf"
server_key_iv: "30a6ec5b8a390316d9d820795bee64b9"
server_key_version: "encryptionkey@0"
I20260808 02:18:31.182153 5016 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:18:31.185457 5037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:31.187331 5016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:31.187491 5016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "4aa9240bda48431ea69e687e2587efe8"
format_stamp: "Formatted at 2026-08-08 02:18:31 on dist-test-slave-1vmg"
server_key: "fd774ae300f53a5ccfb62f365d628acf"
server_key_iv: "30a6ec5b8a390316d9d820795bee64b9"
server_key_version: "encryptionkey@0"
I20260808 02:18:31.187688 5016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:31.213537 5016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:31.217500 5016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:31.217715 5016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:31.218506 5016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:31.219636 5016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:31.219699 5016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:31.219784 5016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:31.219811 5016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:31.230873 5016 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:41903
I20260808 02:18:31.231002 5207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:41903 every 8 connection(s)
I20260808 02:18:31.232036 5016 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:18:31.241528 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 5016
I20260808 02:18:31.241701 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:18:31.242064 24895 external_mini_cluster.cc:1428] Setting key d75d60c92adf1076e59c051c7748a0e5
Aug 08 02:18:31 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155511, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:18:31.249382 4605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:31.234460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:59447 (local address 127.24.79.254:35845)
0808 02:18:31.234602 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:31.234606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:31.235820 (+ 1214us) server_negotiation.cc:407] Connection header received
0808 02:18:31.237325 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:31.237329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:31.237398 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:31.237487 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:31.239013 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:31.239563 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:31.240960 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:31.241234 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:31.241341 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:31.244751 (+ 3410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:31.244754 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:31.244757 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:31.244788 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:31.246633 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:31.247412 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:31.247415 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:31.247416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:31.247519 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:31.247997 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:31.248000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:31.248001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:31.248241 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:31.248345 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:31.249017 (+ 672us) server_negotiation.cc:299] Negotiation successful
0808 02:18:31.249198 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:18:31.250346 5213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:31.234824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.195:59447)
0808 02:18:31.235606 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:31.235664 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:31.237040 (+ 1376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:31.237730 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:31.237739 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:31.238193 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:31.238792 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:31.238811 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:31.239723 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:31.239726 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:31.240791 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:31.240805 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:31.241429 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:31.241434 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:31.241669 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:31.242663 (+ 994us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:31.242689 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:31.244546 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:31.246821 (+ 2275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:31.246829 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:31.246850 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:31.247273 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:31.247671 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:31.247673 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:31.247675 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:31.247876 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:31.248371 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:31.248381 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:31.248723 (+ 342us) client_negotiation.cc:769] Sending connection context
0808 02:18:31.248925 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 02:18:31.249367 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":163,"threads_started":1}
I20260808 02:18:31.252101 5210 heartbeater.cc:347] Connected to a master server at 127.24.79.254:35845
I20260808 02:18:31.252468 5210 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:31.253096 5210 heartbeater.cc:511] Master 127.24.79.254:35845 requested a full tablet report, sending...
I20260808 02:18:31.254499 4545 ts_manager.cc:194] Registered new tserver with Master: 4aa9240bda48431ea69e687e2587efe8 (127.24.79.195:41903)
I20260808 02:18:31.255304 4545 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.195@KRBTEST.COM'} at 127.24.79.195:59447
I20260808 02:18:31.257934 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.79.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.79.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.79.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.193.keytab.
Entry for principal oryx/127.24.79.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.193.keytab.
WARNING: no policy specified for oryx/127.24.79.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.79.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.79.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.194.keytab.
Entry for principal oryx/127.24.79.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.194.keytab.
WARNING: no policy specified for oryx/127.24.79.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.79.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.79.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.195.keytab.
Entry for principal oryx/127.24.79.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.195.keytab.
I20260808 02:18:31.368063 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 4633
I20260808 02:18:31.383074 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 4837
I20260808 02:18:31.396845 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 5016
I20260808 02:18:31.412248 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 4499
I20260808 02:18:31.426273 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:35845
--webserver_interface=127.24.79.254
--webserver_port=43929
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:35845
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:31.566056 5221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:31.566442 5221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:31.566506 5221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:31.571255 5221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:18:31.571359 5221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:31.571388 5221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:31.571410 5221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:18:31.571431 5221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:18:31.577867 5221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:35845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:35845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=43929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:31.579334 5221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:31.580571 5221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:31.590826 5231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:31.590826 5228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:31.590832 5229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:31.591384 5221 server_base.cc:1034] running on GCE node
I20260808 02:18:31.592087 5221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:31.593539 5221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:31.594797 5221 hybrid_clock.cc:648] HybridClock initialized: now 1786155511594769 us; error 55 us; skew 500 ppm
Aug 08 02:18:31 dist-test-slave-1vmg krb5kdc[4482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155511, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:31.598117 5221 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:18:31.599395 5221 webserver.cc:460] Webserver started at http://127.24.79.254:43929/ using document root <none> and password file <none>
I20260808 02:18:31.600080 5221 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:31.600181 5221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:31.605095 5221 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:18:31.607915 5240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:31.609552 5221 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:31.609709 5221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "6d77fc9637e1432aa762f74aefba1843"
format_stamp: "Formatted at 2026-08-08 02:18:30 on dist-test-slave-1vmg"
server_key: "3b62c4a382ac30aa4cdcc1583604d8fc"
server_key_iv: "b21d061e5dbd1c8176c1b10ea7a36867"
server_key_version: "encryptionkey@0"
I20260808 02:18:31.610349 5221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:31.648209 5221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:31.652369 5221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:31.652570 5221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:31.660740 5221 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:35845
I20260808 02:18:31.660789 5323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:35845 every 8 connection(s)
I20260808 02:18:31.662016 5221 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:18:31.666764 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 5221
I20260808 02:18:31.667078 5324 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:18:31.667330 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:36337
--local_ip_for_outbound_sockets=127.24.79.193
--tserver_master_addrs=127.24.79.254:35845
--webserver_port=40723
--webserver_interface=127.24.79.193
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.79.193
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.193.keytab with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:18:31.669140 5324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843: Bootstrap starting.
I20260808 02:18:31.694635 5324 log.cc:824] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:31.704851 5324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:18:31.705384 5324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843: Bootstrap complete.
I20260808 02:18:31.711777 5324 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d77fc9637e1432aa762f74aefba1843" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 35845 } }
I20260808 02:18:31.712386 5324 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d77fc9637e1432aa762f74aefba1843, State: Initialized, Role: FOLLOWER
I20260808 02:18:31.713023 5324 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d77fc9637e1432aa762f74aefba1843" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 35845 } }
I20260808 02:18:31.713191 5324 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:18:31.713266 5324 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:18:31.713362 5324 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:18:31.716547 5324 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d77fc9637e1432aa762f74aefba1843" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 35845 } }
I20260808 02:18:31.716998 5324 leader_election.cc:302] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d77fc9637e1432aa762f74aefba1843; no voters:
I20260808 02:18:31.717414 5324 leader_election.cc:288] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 02:18:31.717615 5329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 02:18:31.717919 5329 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [term 2 LEADER]: Becoming Leader. State: Replica: 6d77fc9637e1432aa762f74aefba1843, State: Running, Role: LEADER
I20260808 02:18:31.718307 5329 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d77fc9637e1432aa762f74aefba1843" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 35845 } }
I20260808 02:18:31.719013 5324 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:18:31.720115 5332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d77fc9637e1432aa762f74aefba1843. Latest consensus state: current_term: 2 leader_uuid: "6d77fc9637e1432aa762f74aefba1843" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d77fc9637e1432aa762f74aefba1843" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 35845 } } }
I20260808 02:18:31.720306 5332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:31.720115 5331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6d77fc9637e1432aa762f74aefba1843" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d77fc9637e1432aa762f74aefba1843" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 35845 } } }
I20260808 02:18:31.720587 5331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:31.720842 5340 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:18:31.724632 5340 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:18:31.725499 5340 catalog_manager.cc:1259] Loaded cluster ID: d8b529e010a342e09e30abbe761a392c
I20260808 02:18:31.725646 5340 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:18:31.728642 5340 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:18:31.729952 5340 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6d77fc9637e1432aa762f74aefba1843: Loaded TSK: 0
I20260808 02:18:31.731658 5340 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 02:18:31.818403 5326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:31.818765 5326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:31.818831 5326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:31.823778 5326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:31.823875 5326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:31.823940 5326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:18:31.829916 5326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.193.keytab
--principal=oryx/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:36337
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=40723
--tserver_master_addrs=127.24.79.254:35845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:31.831557 5326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:31.832793 5326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:31.844537 5359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:31.844985 5361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:31.845233 5326 server_base.cc:1034] running on GCE node
W20260808 02:18:31.845288 5363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:31.845785 5326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:31.846604 5326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:31.847807 5326 hybrid_clock.cc:648] HybridClock initialized: now 1786155511847779 us; error 52 us; skew 500 ppm
Aug 08 02:18:31 dist-test-slave-1vmg krb5kdc[4482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155511, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:31.851341 5326 init.cc:377] Logged in from keytab as oryx/127.24.79.193@KRBTEST.COM (short username oryx)
I20260808 02:18:31.852630 5326 webserver.cc:460] Webserver started at http://127.24.79.193:40723/ using document root <none> and password file <none>
I20260808 02:18:31.853410 5326 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:31.853525 5326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:31.859251 5326 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:18:31.861917 5373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:31.863742 5326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:31.863934 5326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "e8d933c8490743af89d331556c803f83"
format_stamp: "Formatted at 2026-08-08 02:18:30 on dist-test-slave-1vmg"
server_key: "60e95f500eb6d69dd1781d13ba793159"
server_key_iv: "51b94f111f96b5c666873b934cf5eda5"
server_key_version: "encryptionkey@0"
I20260808 02:18:31.864586 5326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:31.884899 5326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:31.888809 5326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:31.889052 5326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:31.889791 5326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:31.890787 5326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:31.890846 5326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:31.890904 5326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:31.890969 5326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260808 02:18:31.901289 5326 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:36337
I20260808 02:18:31.901360 5546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:36337 every 8 connection(s)
I20260808 02:18:31.902460 5326 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:18:31.907791 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 5326
I20260808 02:18:31.908300 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:42051
--local_ip_for_outbound_sockets=127.24.79.194
--tserver_master_addrs=127.24.79.254:35845
--webserver_port=43543
--webserver_interface=127.24.79.194
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.79.194
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.194.keytab with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:18:31 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:31 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 02:18:31.919876 5549 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:31.905256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:46365 (local address 127.24.79.254:35845)
0808 02:18:31.905784 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:31.905812 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:31.906192 (+ 380us) server_negotiation.cc:407] Connection header received
0808 02:18:31.907951 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:31.907967 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:31.908350 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:31.908745 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:31.910176 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:31.911328 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:31.913237 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:31.913894 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:31.914053 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:31.919011 (+ 4958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.193:46365: BlockingRecv error: recv got EOF from 127.24.79.193:46365 (error 108)
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":121,"threads_started":1}
I20260808 02:18:31.919915 5551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:31.905403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.193:46365)
0808 02:18:31.905972 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:31.906024 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:31.907447 (+ 1423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:31.908860 (+ 1413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:31.908870 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:31.909364 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:31.909909 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:31.909934 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:31.911522 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:31.911527 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:31.912950 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:31.912968 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:31.914122 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:31.914127 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:31.914361 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:31.915209 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:31.915236 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:31.918591 (+ 3355us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":176,"threads_started":1}
W20260808 02:18:31.920717 5547 heartbeater.cc:644] Failed to heartbeat to 127.24.79.254:35845 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.79.254:35845: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Aug 08 02:18:31 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:31 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:31.928159 5551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:31.921284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.193:48117)
0808 02:18:31.921498 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:31.921512 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:31.921603 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:31.921882 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:31.921884 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:31.922056 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:31.922178 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:31.922184 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:31.923426 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:31.923433 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:31.924095 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:31.924111 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:31.924831 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:31.924834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:31.924936 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:31.925600 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:31.925615 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:31.928017 (+ 2402us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136}
W20260808 02:18:31.928412 5549 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:31.921424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:48117 (local address 127.24.79.254:35845)
0808 02:18:31.921567 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:31.921573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:31.921592 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:18:31.921689 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:31.921692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:31.921755 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:31.921870 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:31.922311 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:31.923243 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:31.924295 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:31.924666 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:31.924779 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:31.928275 (+ 3496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.193:48117: BlockingRecv error: recv got EOF from 127.24.79.193:48117 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 02:18:31 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:31 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:31.935293 5551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:31.928868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.193:48943)
0808 02:18:31.929076 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:31.929089 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:31.929210 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:31.929468 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:31.929470 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:31.929622 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:31.929716 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:31.929721 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:31.930822 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:31.930826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:31.931491 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:31.931506 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:31.932205 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:31.932209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:31.932289 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:31.932869 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:31.932883 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:31.935150 (+ 2267us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129}
W20260808 02:18:31.935526 5549 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:31.929007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:48943 (local address 127.24.79.254:35845)
0808 02:18:31.929155 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:31.929159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:31.929177 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:18:31.929303 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:31.929306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:31.929358 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:31.929439 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:31.929815 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:31.930665 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:31.931636 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:31.932007 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:31.932131 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:31.935392 (+ 3261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.193:48943: BlockingRecv error: recv got EOF from 127.24.79.193:48943 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 02:18:31.935652 5547 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:18:32.041839 5552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:32.042188 5552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:32.042258 5552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:32.046618 5552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:32.046756 5552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:32.046844 5552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:18:32.053090 5552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.194.keytab
--principal=oryx/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:42051
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=43543
--tserver_master_addrs=127.24.79.254:35845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:32.054677 5552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:32.056212 5552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:32.067039 5564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:32.067039 5558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:32.067055 5559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:32.067471 5552 server_base.cc:1034] running on GCE node
I20260808 02:18:32.067950 5552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:32.068687 5552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:32.069885 5552 hybrid_clock.cc:648] HybridClock initialized: now 1786155512069854 us; error 45 us; skew 500 ppm
Aug 08 02:18:32 dist-test-slave-1vmg krb5kdc[4482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155512, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:32.073254 5552 init.cc:377] Logged in from keytab as oryx/127.24.79.194@KRBTEST.COM (short username oryx)
I20260808 02:18:32.074491 5552 webserver.cc:460] Webserver started at http://127.24.79.194:43543/ using document root <none> and password file <none>
I20260808 02:18:32.075228 5552 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:32.075338 5552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:32.079947 5552 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:32.082351 5573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:32.083963 5552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:18:32.084127 5552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "857f3353aec1432d8244a67f557a599b"
format_stamp: "Formatted at 2026-08-08 02:18:30 on dist-test-slave-1vmg"
server_key: "48a54f6ef10c8b862a3a49bcc224a109"
server_key_iv: "3d856de6574f01897f8475e5aa38a6e3"
server_key_version: "encryptionkey@0"
I20260808 02:18:32.084739 5552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:32.117822 5552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:32.122159 5552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:32.122381 5552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:32.123137 5552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:32.124194 5552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:32.124253 5552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:32.124316 5552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:32.124344 5552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:32.135515 5552 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:42051
I20260808 02:18:32.135607 5736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:42051 every 8 connection(s)
I20260808 02:18:32.136754 5552 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:18:32.138813 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 5552
I20260808 02:18:32.139371 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:41903
--local_ip_for_outbound_sockets=127.24.79.195
--tserver_master_addrs=127.24.79.254:35845
--webserver_port=40065
--webserver_interface=127.24.79.195
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.79.195
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.195.keytab with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:18:32 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:32 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 02:18:32.150763 5549 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:32.139069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:49103 (local address 127.24.79.254:35845)
0808 02:18:32.139350 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:32.139362 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:32.140620 (+ 1258us) server_negotiation.cc:407] Connection header received
0808 02:18:32.142174 (+ 1554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:32.142178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:32.142265 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:32.142351 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:32.143922 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.144481 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.145825 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.146064 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.146168 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:32.150618 (+ 4450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.194:49103: BlockingRecv error: recv got EOF from 127.24.79.194:49103 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 02:18:32.151593 5739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:32.139739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.194:49103)
0808 02:18:32.140386 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:32.140454 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:32.141884 (+ 1430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:32.142607 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:32.142616 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:32.143118 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:32.143723 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:32.143740 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.144646 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.144649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:32.145653 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:32.145666 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.146267 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.146272 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:32.146477 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:32.147331 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:32.147352 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:32.150431 (+ 3079us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":160,"threads_started":1}
W20260808 02:18:32.152182 5737 heartbeater.cc:644] Failed to heartbeat to 127.24.79.254:35845 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.79.254:35845: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Aug 08 02:18:32 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:32 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:32.159245 5739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:32.152719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.194:52335)
0808 02:18:32.152912 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:32.152927 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:32.153020 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:32.153298 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:32.153301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:32.153507 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:32.153650 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:32.153657 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.154500 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.154504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:32.155489 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:32.155502 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.156069 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.156073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:32.156183 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:32.156787 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:32.156804 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:32.159028 (+ 2224us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116}
W20260808 02:18:32.159873 5549 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:32.152868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:52335 (local address 127.24.79.254:35845)
0808 02:18:32.152991 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:32.152995 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:32.153010 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:18:32.153120 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:32.153124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:32.153174 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:32.153258 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:32.153827 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.154380 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.155692 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.155944 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.156044 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:32.159315 (+ 3271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.194:52335: BlockingRecv error: recv got EOF from 127.24.79.194:52335 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 02:18:32 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:32 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:32.166370 5739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:32.160033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.194:54027)
0808 02:18:32.160180 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:32.160196 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:32.160292 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:32.160793 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:32.160796 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:32.160957 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:32.161054 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:32.161060 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.162156 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.162160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:32.162733 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:32.162744 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.163550 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.163553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:32.163734 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:32.164320 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:32.164334 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:32.166246 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 02:18:32.166636 5737 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:18:32.166873 5549 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:32.160354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:54027 (local address 127.24.79.254:35845)
0808 02:18:32.160481 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:32.160486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:32.160500 (+ 14us) server_negotiation.cc:407] Connection header received
0808 02:18:32.160547 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:32.160550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:32.160595 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:32.160673 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:32.161177 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.162006 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.162859 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.163355 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.163491 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:32.166476 (+ 2985us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.194:54027: BlockingRecv error: recv got EOF from 127.24.79.194:54027 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260808 02:18:32.278297 5740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:32.278638 5740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:32.278688 5740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:32.283543 5740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:32.283649 5740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:32.283846 5740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:18:32.290648 5740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/krb5kdc/oryx_127.24.79.195.keytab
--principal=oryx/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:41903
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=40065
--tserver_master_addrs=127.24.79.254:35845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:32.292052 5740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:32.293256 5740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:32.305163 5748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:32.305224 5740 server_base.cc:1034] running on GCE node
W20260808 02:18:32.305226 5752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:32.305848 5749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:32.306499 5740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:32.307412 5740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:32.308670 5740 hybrid_clock.cc:648] HybridClock initialized: now 1786155512308628 us; error 57 us; skew 500 ppm
Aug 08 02:18:32 dist-test-slave-1vmg krb5kdc[4482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155512, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:32.312434 5740 init.cc:377] Logged in from keytab as oryx/127.24.79.195@KRBTEST.COM (short username oryx)
I20260808 02:18:32.313764 5740 webserver.cc:460] Webserver started at http://127.24.79.195:40065/ using document root <none> and password file <none>
I20260808 02:18:32.314438 5740 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:32.314543 5740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:32.319890 5740 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:18:32.322724 5762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:32.324501 5740 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:18:32.324669 5740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "4aa9240bda48431ea69e687e2587efe8"
format_stamp: "Formatted at 2026-08-08 02:18:31 on dist-test-slave-1vmg"
server_key: "fd774ae300f53a5ccfb62f365d628acf"
server_key_iv: "30a6ec5b8a390316d9d820795bee64b9"
server_key_version: "encryptionkey@0"
I20260808 02:18:32.325366 5740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:32.344681 5740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:32.349572 5740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:32.349821 5740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:32.350649 5740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:32.351764 5740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:32.351824 5740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:32.351888 5740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:32.351918 5740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:32.362365 5740 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:41903
I20260808 02:18:32.362408 5928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:41903 every 8 connection(s)
I20260808 02:18:32.363646 5740 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:18:32.369446 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 5740
Aug 08 02:18:32 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:32 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 02:18:32.377779 5549 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:32.365974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:34457 (local address 127.24.79.254:35845)
0808 02:18:32.366117 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:32.366121 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:32.367210 (+ 1089us) server_negotiation.cc:407] Connection header received
0808 02:18:32.368795 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:32.368799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:32.368865 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:32.368984 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:32.370733 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.371451 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.372830 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.373050 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.373142 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:32.377475 (+ 4333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.195:34457: BlockingRecv error: recv got EOF from 127.24.79.195:34457 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:18:32.378304 5931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:32.366372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.195:34457)
0808 02:18:32.367023 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:32.367080 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:32.368491 (+ 1411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:32.369237 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:32.369246 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:32.369719 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:32.370552 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:32.370575 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.371585 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.371588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:32.372685 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:32.372698 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.373263 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.373268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:32.373441 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:32.374139 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:32.374164 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:32.377297 (+ 3133us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":140,"threads_started":1}
W20260808 02:18:32.379042 5929 heartbeater.cc:644] Failed to heartbeat to 127.24.79.254:35845 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.79.254:35845: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
I20260808 02:18:32.381649 5936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:32.370779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49196 (local address 127.24.79.254:35845)
0808 02:18:32.371141 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:32.371145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:32.371159 (+ 14us) server_negotiation.cc:407] Connection header received
0808 02:18:32.371206 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:32.371209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:32.371253 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:32.371346 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:32.372307 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.372840 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.373802 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.374073 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.374184 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:32.375879 (+ 1695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:32.375888 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:32.375918 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:32.375987 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:32.378344 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:32.379472 (+ 1128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:32.379481 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:32.379489 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:32.379710 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:32.380082 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:32.380086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:32.380087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:32.380616 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:32.380868 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:32.381212 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 02:18:32.381497 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":182,"threads_started":1}
Aug 08 02:18:32 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:32 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:32.385757 5931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:32.379569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.195:35087)
0808 02:18:32.379715 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:32.379731 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:32.379854 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:32.380109 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:32.380112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:32.380307 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:32.380432 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:32.380438 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.381399 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.381402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:32.382192 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:32.382206 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.382738 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.382742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:32.382844 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:32.383504 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:32.383518 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:32.385620 (+ 2102us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 02:18:32.385959 5549 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:32.379710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:35087 (local address 127.24.79.254:35845)
0808 02:18:32.379851 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:32.379855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:32.379866 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:18:32.379953 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:32.379955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:32.379995 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:32.380061 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:32.380553 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.381258 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.382378 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.382607 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.382702 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:32.385830 (+ 3128us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.195:35087: BlockingRecv error: recv got EOF from 127.24.79.195:35087 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 02:18:32 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:32 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:32.392848 5931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:32.386348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.195:39117)
0808 02:18:32.386476 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:32.386493 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:32.386577 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:32.386830 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:32.386833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:32.387046 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:32.387191 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:32.387199 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.388204 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.388207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:32.388890 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:32.388903 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.389469 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.389472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:32.389548 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:32.390116 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:32.390131 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:32.392675 (+ 2544us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 02:18:32.393126 5549 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:32.386405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:39117 (local address 127.24.79.254:35845)
0808 02:18:32.386555 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:32.386559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:32.386572 (+ 13us) server_negotiation.cc:407] Connection header received
0808 02:18:32.386686 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:32.386688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:32.386726 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:32.386787 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:32.387413 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.388047 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.389036 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.389314 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.389416 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:32.392911 (+ 3495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.195:39117: BlockingRecv error: recv got EOF from 127.24.79.195:39117 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260808 02:18:32.393368 5929 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 02:18:32 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:32 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:32.943246 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:32.936571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.193:40915)
0808 02:18:32.936897 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:32.936917 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:32.937072 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:32.937411 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:32.937414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:32.937648 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:32.937765 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:32.937777 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.938823 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.938827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:32.939492 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:32.939504 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.940071 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.940075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:32.940187 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:32.940857 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:32.940871 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:32.943095 (+ 2224us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":103,"threads_started":1}
W20260808 02:18:32.943476 5940 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:32.936747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:40915 (local address 127.24.79.254:35845)
0808 02:18:32.937043 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:32.937047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:32.937067 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:18:32.937177 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:32.937179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:32.937228 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:32.937314 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:32.937958 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.938669 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.939648 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:32.939911 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:32.940008 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:32.943320 (+ 3312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.193:40915: BlockingRecv error: recv got EOF from 127.24.79.193:40915 (error 108)
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":118,"threads_started":1}
Aug 08 02:18:33 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:33 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:33.174748 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:33.167533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.194:52775)
0808 02:18:33.167929 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:33.167945 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:33.168072 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:33.168384 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:33.168387 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:33.168635 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:33.168761 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:33.168768 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:33.169629 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:33.169633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:33.170688 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:33.170734 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:33.171385 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:33.171389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:33.171516 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:33.172187 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:33.172205 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:33.174553 (+ 2348us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":102,"threads_started":1}
W20260808 02:18:33.174868 5940 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:33.167603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:52775 (local address 127.24.79.254:35845)
0808 02:18:33.167850 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:33.167854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:33.168033 (+ 179us) server_negotiation.cc:407] Connection header received
0808 02:18:33.168174 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:33.168177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:33.168238 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:33.168317 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:33.168860 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:33.169484 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:33.170886 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:33.171244 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:33.171347 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:33.174752 (+ 3405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.194:52775: BlockingRecv error: recv got EOF from 127.24.79.194:52775 (error 108)
Metrics: {"server-negotiator.queue_time_us":147}
Aug 08 02:18:33 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:33 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:33.400543 5943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:33.394129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.195:46965)
0808 02:18:33.394448 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:33.394462 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:33.394567 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:33.394853 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:33.394856 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:33.395081 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:33.395250 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:33.395256 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:33.396206 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:33.396209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:33.396857 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:33.396869 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:33.397424 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:33.397427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:33.397502 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:33.398047 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:33.398062 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:33.400400 (+ 2338us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":108,"threads_started":1}
W20260808 02:18:33.400727 5940 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:33.394261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:46965 (local address 127.24.79.254:35845)
0808 02:18:33.394399 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:33.394403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:33.394516 (+ 113us) server_negotiation.cc:407] Connection header received
0808 02:18:33.394635 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:33.394638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:33.394686 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:33.394769 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:33.395408 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:33.395994 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:33.397001 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:33.397289 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:33.397386 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:33.400594 (+ 3208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.195:46965: BlockingRecv error: recv got EOF from 127.24.79.195:46965 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 02:18:33 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:33 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:33.951040 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:33.944211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.193:45305)
0808 02:18:33.944523 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:33.944538 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:33.944645 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:33.945107 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:33.945110 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:33.945331 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:33.945450 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:33.945457 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:33.946388 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:33.946391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:33.947074 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:33.947088 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:33.947709 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:33.947713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:33.947794 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:33.948445 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:33.948460 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:33.950809 (+ 2349us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":92,"threads_started":1}
W20260808 02:18:33.951372 5945 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:33.944395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:45305 (local address 127.24.79.254:35845)
0808 02:18:33.944733 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:33.944737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:33.944759 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:18:33.944806 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:33.944808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:33.944853 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:33.944950 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:33.945620 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:33.946220 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:33.947260 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:33.947536 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:33.947644 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:33.951205 (+ 3561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.193:45305: BlockingRecv error: recv got EOF from 127.24.79.193:45305 (error 108)
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":129,"threads_started":1}
Aug 08 02:18:34 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:34 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:34.183190 5946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:34.176059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.194:42813)
0808 02:18:34.176471 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:34.176490 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:34.176630 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:34.177055 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:34.177058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:34.177372 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:34.177493 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:34.177502 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:34.178479 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:34.178483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:34.179320 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:34.179334 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:34.179920 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:34.179924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:34.180003 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:34.180685 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:34.180700 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:34.182965 (+ 2265us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":197,"threads_started":1}
W20260808 02:18:34.183269 5945 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:34.176286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:42813 (local address 127.24.79.254:35845)
0808 02:18:34.176471 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:34.176475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:34.176576 (+ 101us) server_negotiation.cc:407] Connection header received
0808 02:18:34.176801 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:34.176804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:34.176862 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:34.176980 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:34.177628 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:34.178324 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:34.179478 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:34.179743 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:34.179855 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:34.183154 (+ 3299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.194:42813: BlockingRecv error: recv got EOF from 127.24.79.194:42813 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 08 02:18:34 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:34 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:34.407979 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:34.401721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.195:52885)
0808 02:18:34.402071 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:34.402090 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:34.402224 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:34.402495 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:34.402499 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:34.402723 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:34.402848 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:34.402857 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:34.403821 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:34.403826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:34.404593 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:34.404609 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:34.405183 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:34.405186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:34.405265 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:34.405842 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:34.405855 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:34.407809 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":122,"threads_started":1}
W20260808 02:18:34.408206 5945 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:34.402023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:52885 (local address 127.24.79.254:35845)
0808 02:18:34.402194 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:34.402199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:34.402216 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:18:34.402329 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:34.402333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:34.402380 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:34.402451 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:34.403028 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:34.403660 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:34.404745 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:34.404998 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:34.405105 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:34.408058 (+ 2953us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.195:52885: BlockingRecv error: recv got EOF from 127.24.79.195:52885 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 08 02:18:34 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:34 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:34.959036 5949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:34.952136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.193:58899)
0808 02:18:34.952609 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:34.952627 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:34.952751 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:34.953212 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:34.953216 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:34.953425 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:34.953539 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:34.953546 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:34.954595 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:34.954600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:34.955321 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:34.955335 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:34.955842 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:34.955846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:34.955933 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:34.956490 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:34.956503 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:34.958867 (+ 2364us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":100,"threads_started":1}
W20260808 02:18:34.959295 5950 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:34.952370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:58899 (local address 127.24.79.254:35845)
0808 02:18:34.952703 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:34.952707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:34.952724 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:18:34.952932 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:34.952935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:34.952993 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:34.953125 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:34.953757 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:34.954400 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:34.955459 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:34.955711 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:34.955813 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:34.959137 (+ 3324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.193:58899: BlockingRecv error: recv got EOF from 127.24.79.193:58899 (error 108)
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":98,"threads_started":1}
Aug 08 02:18:35 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:35 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:35.190593 5951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:35.184176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.194:42175)
0808 02:18:35.184518 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:35.184536 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:35.184652 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:35.184919 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:35.184922 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:35.185201 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:35.185320 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:35.185328 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:35.186140 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:35.186143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:35.186729 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:35.186739 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:35.187280 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:35.187284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:35.187373 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:35.187990 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:35.188006 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:35.190447 (+ 2441us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":99,"threads_started":1}
W20260808 02:18:35.190846 5950 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:35.184348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:42175 (local address 127.24.79.254:35845)
0808 02:18:35.184528 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:35.184534 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:35.184611 (+ 77us) server_negotiation.cc:407] Connection header received
0808 02:18:35.184754 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:35.184757 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:35.184807 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:35.184877 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:35.185459 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:35.186011 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:35.186862 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:35.187155 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:35.187253 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:35.190715 (+ 3462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.194:42175: BlockingRecv error: recv got EOF from 127.24.79.194:42175 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 08 02:18:35 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:35 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:35.419648 5953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:35.408883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.195:44069)
0808 02:18:35.409339 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:35.409353 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:35.409474 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:35.409752 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:35.409755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:35.409962 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:35.410067 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:35.410074 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:35.411032 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:35.411036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:35.411667 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:35.411681 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:35.412213 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:35.412217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:35.412317 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:35.416922 (+ 4605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:35.416941 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:35.419456 (+ 2515us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":96,"threads_started":1}
W20260808 02:18:35.419821 5950 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:35.408951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:44069 (local address 127.24.79.254:35845)
0808 02:18:35.409090 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:35.409094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:35.409435 (+ 341us) server_negotiation.cc:407] Connection header received
0808 02:18:35.409566 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:35.409569 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:35.409626 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:35.409703 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:35.410193 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:35.410740 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:35.411821 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:35.412079 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:35.412178 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:35.419666 (+ 7488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.195:44069: BlockingRecv error: recv got EOF from 127.24.79.195:44069 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 08 02:18:35 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:35 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:35.966854 5954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:35.960298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.193:42439)
0808 02:18:35.960662 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:35.960681 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:35.960865 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:35.961219 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:35.961222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:35.961428 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:35.961535 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:35.961542 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:35.962410 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:35.962414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:35.963100 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:35.963113 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:35.963629 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:35.963633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:35.963710 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:35.964312 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:35.964325 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:35.966705 (+ 2380us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":96,"threads_started":1}
W20260808 02:18:35.967150 5956 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:35.960423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:42439 (local address 127.24.79.254:35845)
0808 02:18:35.960810 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:35.960814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:35.960831 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:18:35.960985 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:35.960988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:35.961038 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:35.961137 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:35.961694 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:35.962278 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:35.963243 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:35.963482 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:35.963570 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:35.967006 (+ 3436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.193:42439: BlockingRecv error: recv got EOF from 127.24.79.193:42439 (error 108)
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":132,"threads_started":1}
Aug 08 02:18:36 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:36 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:36.198812 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:36.191814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.194:35715)
0808 02:18:36.192184 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:36.192209 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:36.192341 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:36.192637 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:36.192640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:36.192850 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:36.192983 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:36.192992 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:36.194171 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:36.194175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:36.194968 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:36.194981 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:36.195487 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:36.195490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:36.195566 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:36.196233 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:36.196248 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:36.198667 (+ 2419us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":111,"threads_started":1}
W20260808 02:18:36.199318 5956 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:36.192021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:35715 (local address 127.24.79.254:35845)
0808 02:18:36.192201 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:36.192205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:36.192367 (+ 162us) server_negotiation.cc:407] Connection header received
0808 02:18:36.192429 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:36.192432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:36.192489 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:36.192581 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:36.193175 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:36.193924 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:36.195084 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:36.195334 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:36.195430 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:36.198896 (+ 3466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.194:35715: BlockingRecv error: recv got EOF from 127.24.79.194:35715 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 08 02:18:36 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:36 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:36.427096 5958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:36.420923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.195:38733)
0808 02:18:36.421260 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:36.421276 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:36.421411 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:36.421972 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:36.421975 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:36.422249 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:36.422390 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:36.422399 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:36.423350 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:36.423353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:36.423924 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:36.423933 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:36.424440 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:36.424442 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:36.424511 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:36.425063 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:36.425076 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:36.426906 (+ 1830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":100,"threads_started":1}
W20260808 02:18:36.427287 5956 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:36.421022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:38733 (local address 127.24.79.254:35845)
0808 02:18:36.421187 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:36.421192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:36.421358 (+ 166us) server_negotiation.cc:407] Connection header received
0808 02:18:36.421560 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:36.421564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:36.421627 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:36.421711 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:36.422543 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:36.423208 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:36.424058 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:36.424333 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:36.424425 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:36.427150 (+ 2725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.195:38733: BlockingRecv error: recv got EOF from 127.24.79.195:38733 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 08 02:18:36 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:36 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:36.974699 5962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:36.967833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.193:57507)
0808 02:18:36.968181 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:36.968199 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:36.968304 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:36.968623 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:36.968626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:36.968838 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:36.968944 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:36.968951 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:36.969938 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:36.969942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:36.970625 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:36.970637 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:36.971290 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:36.971294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:36.971381 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:36.972002 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:36.972016 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:36.974477 (+ 2461us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":108,"threads_started":1}
W20260808 02:18:36.974834 5963 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:36.967998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:57507 (local address 127.24.79.254:35845)
0808 02:18:36.968255 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:36.968260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:36.968278 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:18:36.968399 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:36.968402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:36.968449 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:36.968536 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:36.969092 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:36.969782 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:36.970807 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:36.971102 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:36.971204 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:36.974701 (+ 3497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.193:57507: BlockingRecv error: recv got EOF from 127.24.79.193:57507 (error 108)
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":85,"threads_started":1}
Aug 08 02:18:37 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:37 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:37.207260 5966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:37.199907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.194:43585)
0808 02:18:37.200656 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:37.200676 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:37.200788 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:37.201098 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:37.201101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:37.201351 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:37.201458 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:37.201464 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:37.202558 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:37.202561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:37.203252 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:37.203265 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:37.203858 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:37.203862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:37.203942 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:37.204630 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:37.204646 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:37.207094 (+ 2448us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":85,"threads_started":1}
W20260808 02:18:37.207559 5963 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:37.200545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:43585 (local address 127.24.79.254:35845)
0808 02:18:37.200818 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:37.200822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:37.200837 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:18:37.200883 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:37.200885 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:37.200928 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:37.201015 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:37.201726 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:37.202389 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:37.203416 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:37.203673 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:37.203770 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:37.207428 (+ 3658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.194:43585: BlockingRecv error: recv got EOF from 127.24.79.194:43585 (error 108)
Metrics: {"server-negotiator.queue_time_us":165}
Aug 08 02:18:37 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:37 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:37.434723 5967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:37.428043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.195:54217)
0808 02:18:37.428372 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:37.428386 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:37.428497 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:37.428823 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:37.428826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:37.429040 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:37.429205 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:37.429212 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:37.430125 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:37.430128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:37.431013 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:37.431041 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:37.431691 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:37.431694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:37.431775 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:37.432515 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:37.432530 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:37.434575 (+ 2045us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":122,"threads_started":1}
W20260808 02:18:37.435006 5963 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:37.428223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:54217 (local address 127.24.79.254:35845)
0808 02:18:37.428368 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:37.428372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:37.428444 (+ 72us) server_negotiation.cc:407] Connection header received
0808 02:18:37.428572 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:37.428575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:37.428625 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:37.428708 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:37.429344 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:37.429992 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:37.431257 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:37.431509 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:37.431615 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:37.434832 (+ 3217us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.195:54217: BlockingRecv error: recv got EOF from 127.24.79.195:54217 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 02:18:37 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:37 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:37.982486 5972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:37.975958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.193:59973)
0808 02:18:37.976306 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:37.976335 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:37.976454 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:37.976743 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:37.976745 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:37.976950 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:37.977052 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:37.977058 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:37.978078 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:37.978081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:37.978840 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:37.978852 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:37.979573 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:37.979577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:37.979663 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:37.980247 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:37.980260 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:37.982339 (+ 2079us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":95,"threads_started":1}
W20260808 02:18:37.982750 5973 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:37.976125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:59973 (local address 127.24.79.254:35845)
0808 02:18:37.976434 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:37.976438 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:37.976455 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:18:37.976549 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:37.976552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:37.976602 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:37.976695 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:37.977377 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:37.977945 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:37.979121 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:37.979424 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:37.979523 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:37.982594 (+ 3071us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.193:59973: BlockingRecv error: recv got EOF from 127.24.79.193:59973 (error 108)
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":92,"threads_started":1}
Aug 08 02:18:38 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:38 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:38.214481 5974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:38.208292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.194:38125)
0808 02:18:38.208648 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:38.208666 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:38.208842 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:38.209110 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:38.209113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:38.209369 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:38.209475 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:38.209482 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:38.210293 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:38.210297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:38.210873 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:38.210883 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:38.211406 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:38.211411 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:38.211501 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:38.212066 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:38.212079 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:38.214282 (+ 2203us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":127,"threads_started":1}
W20260808 02:18:38.214607 5973 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:38.208455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:38125 (local address 127.24.79.254:35845)
0808 02:18:38.208650 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:38.208655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:38.208784 (+ 129us) server_negotiation.cc:407] Connection header received
0808 02:18:38.208943 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:38.208947 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:38.208995 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:38.209066 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:38.209613 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:38.210156 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:38.211030 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:38.211279 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:38.211375 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:38.214495 (+ 3120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.194:38125: BlockingRecv error: recv got EOF from 127.24.79.194:38125 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 08 02:18:38 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:38 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:38.442080 5975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:38.435688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.195:34747)
0808 02:18:38.435994 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:38.436008 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:38.436136 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:38.436469 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:38.436472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:38.436687 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:38.436800 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:38.436806 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:38.437684 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:38.437688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:38.438336 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:38.438348 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:38.438961 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:38.438965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:38.439056 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:38.439686 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:38.439698 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:38.441939 (+ 2241us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":86,"threads_started":1}
W20260808 02:18:38.442327 5973 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:38.435809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:34747 (local address 127.24.79.254:35845)
0808 02:18:38.435969 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:38.435973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:38.436173 (+ 200us) server_negotiation.cc:407] Connection header received
0808 02:18:38.436232 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:38.436235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:38.436290 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:38.436376 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:38.436968 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:38.437534 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:38.438488 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:38.438769 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:38.438865 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:38.442192 (+ 3327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.195:34747: BlockingRecv error: recv got EOF from 127.24.79.195:34747 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 08 02:18:38 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:38 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:38.990561 5976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:38.983568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.193:48911)
0808 02:18:38.983996 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:38.984012 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:38.984141 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:38.984443 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:38.984446 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:38.984649 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:38.984756 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:38.984761 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:38.985706 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:38.985710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:38.986341 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:38.986353 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:38.986994 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:38.986999 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:38.987136 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:38.987935 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:38.987951 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:38.990427 (+ 2476us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":105,"threads_started":1}
W20260808 02:18:38.990792 5977 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:38.983694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:48911 (local address 127.24.79.254:35845)
0808 02:18:38.983967 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:38.983971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:38.984171 (+ 200us) server_negotiation.cc:407] Connection header received
0808 02:18:38.984266 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:38.984268 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:38.984322 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:38.984412 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:38.984909 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:38.985551 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:38.986498 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:38.986775 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:38.986873 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:38.990601 (+ 3728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.193:48911: BlockingRecv error: recv got EOF from 127.24.79.193:48911 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":71,"threads_started":1}
Aug 08 02:18:39 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:39 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:39.222543 5978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:39.215599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.194:60435)
0808 02:18:39.215983 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:39.215999 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:39.216131 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:39.216570 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:39.216573 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:39.216838 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:39.216948 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:39.216955 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:39.217818 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:39.217821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:39.218422 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:39.218432 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:39.219065 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:39.219069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:39.219152 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:39.219802 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:39.219815 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:39.222388 (+ 2573us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":118,"threads_started":1}
W20260808 02:18:39.222754 5977 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:39.215695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:60435 (local address 127.24.79.254:35845)
0808 02:18:39.215843 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:39.215847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:39.216070 (+ 223us) server_negotiation.cc:407] Connection header received
0808 02:18:39.216235 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:39.216238 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:39.216297 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:39.216539 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:39.217105 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:39.217672 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:39.218610 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:39.218862 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:39.219012 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:39.222614 (+ 3602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.194:60435: BlockingRecv error: recv got EOF from 127.24.79.194:60435 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 02:18:39 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:39 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:39.448835 5979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:39.443048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.195:33075)
0808 02:18:39.443413 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:39.443429 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:39.443541 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:39.443840 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:39.443843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:39.444050 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:39.444156 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:39.444161 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:39.444948 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:39.444950 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:39.445559 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:39.445568 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:39.445996 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:39.445998 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:39.446063 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:39.446677 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:39.446691 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:39.448706 (+ 2015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":113,"threads_started":1}
W20260808 02:18:39.449014 5977 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:39.443138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:33075 (local address 127.24.79.254:35845)
0808 02:18:39.443281 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:39.443286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:39.443530 (+ 244us) server_negotiation.cc:407] Connection header received
0808 02:18:39.443618 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:39.443625 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:39.443677 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:39.443758 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:39.444292 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:39.444859 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:39.445669 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:39.445899 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:39.445981 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:39.448887 (+ 2906us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.195:33075: BlockingRecv error: recv got EOF from 127.24.79.195:33075 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 02:18:39 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:39 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:39.997925 5980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:39.991573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.193:55657)
0808 02:18:39.991939 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:39.991953 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:39.992063 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:39.992373 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:39.992375 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:39.992571 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:39.992717 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:39.992723 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:39.993578 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:39.993582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:39.994268 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:39.994279 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:39.994832 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:39.994835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:39.994974 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:39.995497 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:39.995509 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:39.997787 (+ 2278us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":166,"threads_started":1}
W20260808 02:18:39.998183 5981 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:39.991807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:55657 (local address 127.24.79.254:35845)
0808 02:18:39.992122 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:39.992126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:39.992146 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:18:39.992203 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:39.992205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:39.992250 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:39.992338 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:39.992860 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:39.993451 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:39.994430 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:39.994705 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:39.994821 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:39.997979 (+ 3158us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.193:55657: BlockingRecv error: recv got EOF from 127.24.79.193:55657 (error 108)
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":68,"threads_started":1}
Aug 08 02:18:40 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:40 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:40.229744 5982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:40.223508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.194:48237)
0808 02:18:40.223808 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:40.223823 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:40.223958 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:40.224318 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:40.224321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:40.224535 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:40.224642 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:40.224649 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:40.225637 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:40.225640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:40.226237 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:40.226246 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:40.226816 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:40.226819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:40.226899 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:40.227526 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:40.227540 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:40.229608 (+ 2068us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":96,"threads_started":1}
W20260808 02:18:40.230023 5981 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:40.223544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:48237 (local address 127.24.79.254:35845)
0808 02:18:40.223689 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:40.223694 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:40.223919 (+ 225us) server_negotiation.cc:407] Connection header received
0808 02:18:40.224111 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:40.224114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:40.224178 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:40.224266 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:40.224830 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:40.225516 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:40.226385 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:40.226633 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:40.226733 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:40.229889 (+ 3156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.194:48237: BlockingRecv error: recv got EOF from 127.24.79.194:48237 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 08 02:18:40 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:40 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:40.462273 5984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:40.449878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.195:56693)
0808 02:18:40.450280 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:40.450293 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:40.450418 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:40.450747 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:40.450750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:40.451034 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:40.451191 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:40.451198 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:40.451977 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:40.451980 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:40.452548 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:40.452557 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:40.453037 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:40.453041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:40.453146 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:40.459619 (+ 6473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:40.459637 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:40.462117 (+ 2480us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":88,"threads_started":1}
W20260808 02:18:40.462509 5981 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:40.450080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:56693 (local address 127.24.79.254:35845)
0808 02:18:40.450259 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:40.450264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:40.450384 (+ 120us) server_negotiation.cc:407] Connection header received
0808 02:18:40.450518 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:40.450521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:40.450612 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:40.450684 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:40.451322 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:40.451855 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:40.452681 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:40.452921 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:40.453016 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:40.462374 (+ 9358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.195:56693: BlockingRecv error: recv got EOF from 127.24.79.195:56693 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 02:18:41 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:41 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:41.007058 5985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:40.999130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.193:35911)
0808 02:18:40.999476 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:40.999493 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:40.999615 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:40.999988 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:40.999991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:41.000235 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:41.000435 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:41.000445 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:41.001398 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:41.001402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:41.001983 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:41.001994 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:41.002750 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:41.002756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:41.002862 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:41.003668 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:41.003683 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:41.006824 (+ 3141us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":105,"threads_started":1}
W20260808 02:18:41.007414 5986 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:40.999328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:35911 (local address 127.24.79.254:35845)
0808 02:18:40.999572 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:40.999577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:40.999594 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:18:40.999745 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:40.999748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:40.999810 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:40.999900 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:41.000644 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:41.001236 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:41.002230 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:41.002558 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:41.002662 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:41.007145 (+ 4483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.193:35911: BlockingRecv error: recv got EOF from 127.24.79.193:35911 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":80,"threads_started":1}
Aug 08 02:18:41 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:41 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:41.237814 5987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:41.230893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.194:35401)
0808 02:18:41.231201 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:41.231216 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:41.231348 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:41.231798 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:41.231802 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:41.232143 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:41.232310 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:41.232318 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:41.233336 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:41.233339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:41.233924 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:41.233934 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:41.234502 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:41.234505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:41.234599 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:41.235260 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:41.235278 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:41.237619 (+ 2341us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":96,"threads_started":1}
W20260808 02:18:41.238008 5986 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:41.231014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:35401 (local address 127.24.79.254:35845)
0808 02:18:41.231172 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:41.231177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:41.231299 (+ 122us) server_negotiation.cc:407] Connection header received
0808 02:18:41.231497 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:41.231502 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:41.231607 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:41.231693 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:41.232457 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:41.233185 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:41.234083 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:41.234346 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:41.234440 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:41.237876 (+ 3436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.194:35401: BlockingRecv error: recv got EOF from 127.24.79.194:35401 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 02:18:41 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:41 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:41.469998 5988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:41.463218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.195:50469)
0808 02:18:41.463532 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:41.463546 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:41.463659 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:41.463961 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:41.463964 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:41.464202 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:41.464314 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:41.464320 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:41.465211 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:41.465214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:41.465803 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:41.465814 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:41.466402 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:41.466405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:41.466489 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:41.467217 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:41.467231 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:41.469814 (+ 2583us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":108,"threads_started":1}
W20260808 02:18:41.470124 5986 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:41.463358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:50469 (local address 127.24.79.254:35845)
0808 02:18:41.463498 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:41.463501 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:41.463659 (+ 158us) server_negotiation.cc:407] Connection header received
0808 02:18:41.463748 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:41.463751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:41.463806 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:41.463871 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:41.464474 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:41.465030 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:41.465954 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:41.466247 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:41.466342 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:41.470013 (+ 3671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.195:50469: BlockingRecv error: recv got EOF from 127.24.79.195:50469 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 02:18:42 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:42 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:42.015770 5989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:42.008154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.193:37439)
0808 02:18:42.008594 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:42.008610 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:42.008722 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:42.009488 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:42.009491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:42.009751 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:42.009868 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:42.009876 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:42.010748 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:42.010751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:42.011417 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:42.011429 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:42.011998 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:42.012002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:42.012080 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:42.012998 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:42.013013 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:42.015620 (+ 2607us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":88,"threads_started":1}
W20260808 02:18:42.016023 5991 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:42.008894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:37439 (local address 127.24.79.254:35845)
0808 02:18:42.009165 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:42.009171 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:42.009188 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:18:42.009233 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:42.009236 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:42.009286 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:42.009385 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:42.010017 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:42.010600 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:42.011577 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:42.011838 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:42.011951 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:42.015866 (+ 3915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.193:37439: BlockingRecv error: recv got EOF from 127.24.79.193:37439 (error 108)
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":87,"threads_started":1}
Aug 08 02:18:42 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:42 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:42.245581 5992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:42.239014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.194:46311)
0808 02:18:42.239390 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:42.239405 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:42.239539 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:42.239820 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:42.239823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:42.240045 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:42.240148 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:42.240155 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:42.241103 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:42.241108 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:42.241746 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:42.241759 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:42.242266 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:42.242270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:42.242359 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:42.242988 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:42.243002 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:42.245408 (+ 2406us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":112,"threads_started":1}
W20260808 02:18:42.245710 5991 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:42.239158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:46311 (local address 127.24.79.254:35845)
0808 02:18:42.239365 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:42.239370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:42.239488 (+ 118us) server_negotiation.cc:407] Connection header received
0808 02:18:42.239646 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:42.239649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:42.239697 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:42.239780 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:42.240277 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:42.240851 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:42.241890 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:42.242139 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:42.242240 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:42.245603 (+ 3363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.194:46311: BlockingRecv error: recv got EOF from 127.24.79.194:46311 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 08 02:18:42 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:42 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:42.478319 5994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:42.471418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.195:58041)
0808 02:18:42.471848 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:42.471873 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:42.472009 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:42.472394 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:42.472397 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:42.472604 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:42.472713 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:42.472719 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:42.473700 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:42.473705 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:42.474466 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:42.474481 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:42.475091 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:42.475095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:42.475207 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:42.475740 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:42.475752 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:42.478142 (+ 2390us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":97,"threads_started":1}
W20260808 02:18:42.478470 5991 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:42.471673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:58041 (local address 127.24.79.254:35845)
0808 02:18:42.471836 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:42.471840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:42.472003 (+ 163us) server_negotiation.cc:407] Connection header received
0808 02:18:42.472183 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:42.472186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:42.472240 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:42.472336 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:42.472871 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:42.473532 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:42.474618 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:42.474900 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:42.475051 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:42.478350 (+ 3299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.195:58041: BlockingRecv error: recv got EOF from 127.24.79.195:58041 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 08 02:18:43 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:43 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:43.023818 5996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:43.016858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.193:45365)
0808 02:18:43.017220 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:43.017236 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:43.017348 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:43.017722 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:43.017726 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:43.017930 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:43.018037 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:43.018058 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:43.018995 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:43.018999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:43.019597 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:43.019608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:43.020138 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:43.020141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:43.020212 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:43.020996 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:43.021012 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:43.023680 (+ 2668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":84,"threads_started":1}
W20260808 02:18:43.024049 5997 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:43.016984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:45365 (local address 127.24.79.254:35845)
0808 02:18:43.017281 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:43.017284 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:43.017301 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:18:43.017484 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:43.017488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:43.017542 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:43.017653 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:43.018198 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:43.018820 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:43.019715 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:43.020017 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:43.020129 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:43.023911 (+ 3782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.193:45365: BlockingRecv error: recv got EOF from 127.24.79.193:45365 (error 108)
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":90,"threads_started":1}
Aug 08 02:18:43 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:43 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:43.253557 6000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:43.246566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.194:60699)
0808 02:18:43.246908 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:43.246947 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:43.247054 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:43.247418 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:43.247420 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:43.247658 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:43.247772 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:43.247778 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:43.248717 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:43.248721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:43.249383 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:43.249396 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:43.249987 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:43.249991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:43.250070 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:43.250783 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:43.250798 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:43.253325 (+ 2527us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":82,"threads_started":1}
W20260808 02:18:43.253664 5997 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:43.246783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:60699 (local address 127.24.79.254:35845)
0808 02:18:43.246982 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:43.246986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:43.247002 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:18:43.247198 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:43.247201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:43.247301 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:43.247384 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:43.247921 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:43.248546 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:43.249568 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:43.249826 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:43.249925 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:43.253537 (+ 3612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.194:60699: BlockingRecv error: recv got EOF from 127.24.79.194:60699 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 08 02:18:43 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:43 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:43.485563 6001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:43.479487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.195:46849)
0808 02:18:43.479821 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:43.479838 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:43.479973 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:43.480239 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:43.480242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:43.480455 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:43.480582 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:43.480588 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:43.481407 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:43.481411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:43.481971 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:43.481979 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:43.482463 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:43.482467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:43.482558 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:43.483232 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:43.483250 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:43.485422 (+ 2172us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":97,"threads_started":1}
W20260808 02:18:43.485760 5997 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:43.479630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:46849 (local address 127.24.79.254:35845)
0808 02:18:43.479821 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:43.479826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:43.479918 (+ 92us) server_negotiation.cc:407] Connection header received
0808 02:18:43.480070 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:43.480073 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:43.480126 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:43.480198 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:43.480722 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:43.481283 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:43.482090 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:43.482320 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:43.482410 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:43.485622 (+ 3212us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.195:46849: BlockingRecv error: recv got EOF from 127.24.79.195:46849 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 02:18:44 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:44 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:44.031392 6002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:44.024806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.193:45405)
0808 02:18:44.025109 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:44.025127 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:44.025229 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:44.025637 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:44.025641 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:44.025844 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:44.025957 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:44.025964 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:44.026901 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:44.026905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:44.027519 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:44.027531 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:44.028111 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:44.028116 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:44.028212 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:44.028802 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:44.028816 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:44.031205 (+ 2389us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":88,"threads_started":1}
W20260808 02:18:44.031531 6003 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:44.024914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:45405 (local address 127.24.79.254:35845)
0808 02:18:44.025188 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:44.025193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:44.025214 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:18:44.025383 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:44.025386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:44.025440 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:44.025535 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:44.026155 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:44.026735 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:44.027666 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:44.027948 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:44.028052 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:44.031402 (+ 3350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.193:45405: BlockingRecv error: recv got EOF from 127.24.79.193:45405 (error 108)
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":88,"threads_started":1}
Aug 08 02:18:44 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:44 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:44.260810 6004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:44.254568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.194:44543)
0808 02:18:44.254900 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:44.254943 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:44.255073 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:44.255356 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:44.255359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:44.255576 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:44.255679 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:44.255685 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:44.256483 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:44.256487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:44.257070 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:44.257080 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:44.257565 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:44.257569 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:44.257659 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:44.258347 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:44.258360 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:44.260671 (+ 2311us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":122,"threads_started":1}
W20260808 02:18:44.261066 6003 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:44.254690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:44543 (local address 127.24.79.254:35845)
0808 02:18:44.254846 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:44.254853 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:44.255026 (+ 173us) server_negotiation.cc:407] Connection header received
0808 02:18:44.255167 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:44.255170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:44.255225 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:44.255302 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:44.255812 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:44.256362 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:44.257206 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:44.257441 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:44.257537 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:44.260929 (+ 3392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.194:44543: BlockingRecv error: recv got EOF from 127.24.79.194:44543 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 02:18:44 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:44 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:44.493834 6007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:44.486676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.195:45223)
0808 02:18:44.487044 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:44.487060 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:44.487250 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:44.487577 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:44.487580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:44.487791 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:44.487909 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:44.487915 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:44.488911 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:44.488914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:44.489545 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:44.489560 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:44.490241 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:44.490245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:44.490337 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:44.491085 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:44.491100 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:44.493691 (+ 2591us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":92,"threads_started":1}
W20260808 02:18:44.494022 6003 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:44.486788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:45223 (local address 127.24.79.254:35845)
0808 02:18:44.486990 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:44.486995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:44.487227 (+ 232us) server_negotiation.cc:407] Connection header received
0808 02:18:44.487356 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:44.487359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:44.487412 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:44.487479 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:44.488101 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:44.488731 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:44.489741 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:44.490020 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:44.490151 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:44.493915 (+ 3764us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.195:45223: BlockingRecv error: recv got EOF from 127.24.79.195:45223 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 02:18:45 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:45 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:45.038949 6008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:45.032434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.193:46253)
0808 02:18:45.032750 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:45.032764 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:45.032864 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:45.033212 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:45.033215 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:45.033426 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:45.033536 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:45.033542 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:45.034493 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:45.034498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:45.035196 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:45.035211 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:45.035775 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:45.035778 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:45.035862 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:45.036490 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:45.036504 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:45.038761 (+ 2257us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":137,"threads_started":1}
W20260808 02:18:45.039172 6009 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:45.032570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:46253 (local address 127.24.79.254:35845)
0808 02:18:45.032816 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:45.032821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:45.032836 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:18:45.032946 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:45.032949 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:45.033000 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:45.033095 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:45.033706 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:45.034344 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:45.035361 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:45.035618 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:45.035719 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:45.039023 (+ 3304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.193:46253: BlockingRecv error: recv got EOF from 127.24.79.193:46253 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":80,"threads_started":1}
Aug 08 02:18:45 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:45 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:45.268520 6011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:45.261926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.194:50843)
0808 02:18:45.262253 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:45.262272 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:45.262382 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:45.262702 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:45.262705 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:45.262908 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:45.263071 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:45.263077 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:45.264061 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:45.264066 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:45.264710 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:45.264722 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:45.265333 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:45.265337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:45.265421 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:45.266087 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:45.266101 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:45.268357 (+ 2256us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":137,"threads_started":1}
W20260808 02:18:45.268707 6009 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:45.262223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:50843 (local address 127.24.79.254:35845)
0808 02:18:45.262381 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:45.262384 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:45.262400 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:18:45.262476 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:45.262479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:45.262525 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:45.262596 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:45.263207 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:45.263818 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:45.264888 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:45.265143 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:45.265302 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:45.268562 (+ 3260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.194:50843: BlockingRecv error: recv got EOF from 127.24.79.194:50843 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 02:18:45 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:45 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:45.505638 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:45.495237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.195:47121)
0808 02:18:45.495622 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:45.495640 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:45.495773 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:45.496080 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:45.496083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:45.496310 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:45.496429 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:45.496436 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:45.497304 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:45.497308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:45.497902 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:45.497913 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:45.498679 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:45.498684 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:45.498771 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:45.503103 (+ 4332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:45.503121 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:45.505430 (+ 2309us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":100,"threads_started":1}
W20260808 02:18:45.505759 6009 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:45.495388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:47121 (local address 127.24.79.254:35845)
0808 02:18:45.495570 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:45.495576 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:45.495760 (+ 184us) server_negotiation.cc:407] Connection header received
0808 02:18:45.495869 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:45.495871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:45.495928 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:45.495993 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:45.496584 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:45.497148 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:45.498118 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:45.498413 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:45.498536 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:45.505639 (+ 7103us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.195:47121: BlockingRecv error: recv got EOF from 127.24.79.195:47121 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 08 02:18:46 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:46 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:46.046679 6017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:46.039897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.193:34891)
0808 02:18:46.040283 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:46.040298 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:46.040402 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:46.040767 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:46.040770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:46.040993 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:46.041108 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:46.041114 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:46.041998 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:46.042002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:46.042775 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:46.042791 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:46.043419 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:46.043422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:46.043501 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:46.044105 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:46.044120 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:46.046513 (+ 2393us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":103,"threads_started":1}
W20260808 02:18:46.046957 6018 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:46.039997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:34891 (local address 127.24.79.254:35845)
0808 02:18:46.040348 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:46.040351 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:46.040366 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:18:46.040532 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:46.040536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:46.040592 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:46.040688 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:46.041299 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:46.041859 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:46.042997 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:46.043267 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:46.043362 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:46.046756 (+ 3394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.193:34891: BlockingRecv error: recv got EOF from 127.24.79.193:34891 (error 108)
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":73,"threads_started":1}
Aug 08 02:18:46 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:46 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:46.276501 6019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:46.269676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.194:37417)
0808 02:18:46.270119 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:46.270150 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:46.270291 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:46.270609 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:46.270612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:46.270827 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:46.271008 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:46.271016 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:46.271967 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:46.271971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:46.272690 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:46.272704 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:46.273348 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:46.273351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:46.273453 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:46.274076 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:46.274091 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:46.276349 (+ 2258us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":113,"threads_started":1}
W20260808 02:18:46.276759 6018 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:46.269867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:37417 (local address 127.24.79.254:35845)
0808 02:18:46.270164 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:46.270171 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:46.270277 (+ 106us) server_negotiation.cc:407] Connection header received
0808 02:18:46.270378 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:46.270380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:46.270435 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:46.270517 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:46.271221 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:46.271806 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:46.272907 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:46.273166 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:46.273354 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:46.276624 (+ 3270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.194:37417: BlockingRecv error: recv got EOF from 127.24.79.194:37417 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 02:18:46 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:46 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.195@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:46.513283 6020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:46.506762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.195:38005)
0808 02:18:46.507079 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:46.507096 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:46.507254 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:46.507556 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:46.507559 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:46.507791 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:46.507996 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:46.508018 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:46.508890 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:46.508894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:46.509472 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:46.509481 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:46.509958 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:46.509962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:46.510048 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:46.510643 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:46.510657 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:46.513112 (+ 2455us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":104,"threads_started":1}
W20260808 02:18:46.513629 6018 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:46.506884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:38005 (local address 127.24.79.254:35845)
0808 02:18:46.507079 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:46.507084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:46.507246 (+ 162us) server_negotiation.cc:407] Connection header received
0808 02:18:46.507360 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:46.507365 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:46.507427 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:46.507508 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:46.508140 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:46.508775 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:46.509596 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:46.509833 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:46.509948 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:46.513497 (+ 3549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.195:38005: BlockingRecv error: recv got EOF from 127.24.79.195:38005 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 08 02:18:47 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:47 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.193@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:47.054679 6021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:47.048074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.193:50375)
0808 02:18:47.048458 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:47.048473 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:47.048583 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:47.048953 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:47.048956 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:47.049158 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:47.049326 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:47.049334 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:47.050213 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:47.050218 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:47.050834 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:47.050846 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:47.051455 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:47.051459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:47.051543 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:47.052271 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:47.052286 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:47.054523 (+ 2237us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":112,"threads_started":1}
W20260808 02:18:47.055154 6022 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:47.048264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:50375 (local address 127.24.79.254:35845)
0808 02:18:47.048524 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:47.048528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:47.048544 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:18:47.048732 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:47.048736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:47.048793 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:47.048887 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:47.049491 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:47.050059 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:47.051011 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:47.051299 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:47.051402 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:47.055028 (+ 3626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.193:50375: BlockingRecv error: recv got EOF from 127.24.79.193:50375 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":72,"threads_started":1}
Aug 08 02:18:47 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:18:47 dist-test-slave-1vmg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.79.194@KRBTEST.COM for oryx/127.24.79.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:18:47.284638 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:47.277711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:35845 (local address 127.24.79.194:35087)
0808 02:18:47.278052 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:47.278070 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:47.278184 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:47.278622 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:47.278626 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:47.278892 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:47.279059 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:47.279068 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:47.280052 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:47.280056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:47.280676 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:47.280688 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:47.281274 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:47.281278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:47.281359 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:47.281992 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:47.282010 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:47.284463 (+ 2453us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.79.254:35845: Server oryx/127.24.79.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":108,"threads_started":1}
W20260808 02:18:47.284842 6022 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:47.277963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:35087 (local address 127.24.79.254:35845)
0808 02:18:47.278284 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:47.278290 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:47.278310 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:18:47.278365 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:47.278369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:47.278423 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:47.278531 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:47.279286 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:47.279903 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:47.280857 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:47.281112 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:47.281211 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:47.284682 (+ 3471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.79.194:35087: BlockingRecv error: recv got EOF from 127.24.79.194:35087 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
I20260808 02:18:47.372538 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 5326
I20260808 02:18:47.386322 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 5552
I20260808 02:18:47.410360 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 5740
I20260808 02:18:47.422601 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 5221
2026-08-08T02:18:47Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18642 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T02:18:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:18:47Z Disabled control of system clock
I20260808 02:18:47.462193 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:42389
--webserver_interface=127.24.79.254
--webserver_port=0
--builtin_ntp_servers=127.24.79.212:33149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:42389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 02:18:47.598624 6028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:47.599014 6028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:47.599068 6028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:47.603355 6028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:18:47.603443 6028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:47.603458 6028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:47.603474 6028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:18:47.603489 6028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:18:47.608672 6028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:33149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:42389
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:42389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:47.609921 6028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:47.611495 6028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:47.622301 6039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:47.622278 6038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:47.622475 6028 server_base.cc:1034] running on GCE node
W20260808 02:18:47.622241 6041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:47.623457 6028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:47.624861 6028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:47.626072 6028 hybrid_clock.cc:648] HybridClock initialized: now 1786155527626054 us; error 68 us; skew 500 ppm
I20260808 02:18:47.628759 6028 webserver.cc:460] Webserver started at http://127.24.79.254:39889/ using document root <none> and password file <none>
I20260808 02:18:47.629493 6028 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:47.629586 6028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:47.629827 6028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:47.632359 6028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "8702d46265dd4ab7be55d021e7de88b0"
format_stamp: "Formatted at 2026-08-08 02:18:47 on dist-test-slave-1vmg"
server_key: "23796b580df775792f866dd2a6e44ffd"
server_key_iv: "eeec980ccc4162ec34646ec5351c8e1d"
server_key_version: "encryptionkey@0"
I20260808 02:18:47.633008 6028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "8702d46265dd4ab7be55d021e7de88b0"
format_stamp: "Formatted at 2026-08-08 02:18:47 on dist-test-slave-1vmg"
server_key: "23796b580df775792f866dd2a6e44ffd"
server_key_iv: "eeec980ccc4162ec34646ec5351c8e1d"
server_key_version: "encryptionkey@0"
I20260808 02:18:47.637862 6028 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 02:18:47.641443 6050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:47.643406 6028 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:18:47.643601 6028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "8702d46265dd4ab7be55d021e7de88b0"
format_stamp: "Formatted at 2026-08-08 02:18:47 on dist-test-slave-1vmg"
server_key: "23796b580df775792f866dd2a6e44ffd"
server_key_iv: "eeec980ccc4162ec34646ec5351c8e1d"
server_key_version: "encryptionkey@0"
I20260808 02:18:47.643749 6028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:47.659032 6028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:47.659893 6028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:47.660110 6028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:47.669044 6028 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:42389
I20260808 02:18:47.669083 6128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:42389 every 8 connection(s)
I20260808 02:18:47.670194 6028 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:18:47.671630 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 6028
I20260808 02:18:47.671746 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:18:47.672037 24895 external_mini_cluster.cc:1428] Setting key 0953417227dd5f5305ac47f88cce65d7
I20260808 02:18:47.674512 6130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:47.682194 6130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8702d46265dd4ab7be55d021e7de88b0: Bootstrap starting.
I20260808 02:18:47.683351 6138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:47.673598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44960 (local address 127.24.79.254:42389)
0808 02:18:47.674451 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:47.674473 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:47.674517 (+ 44us) server_negotiation.cc:407] Connection header received
0808 02:18:47.675318 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:47.675341 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:47.675806 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:47.676156 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:47.676748 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:47.677834 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:47.679145 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:47.679676 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:47.679827 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:47.680967 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:47.680973 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:47.680991 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:18:47.681479 (+ 488us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:47.681554 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:18:47.681578 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:47.681672 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:47.681919 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 02:18:47.682363 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":166,"threads_started":1}
I20260808 02:18:47.684903 6130 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8702d46265dd4ab7be55d021e7de88b0: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:47.685937 6130 log.cc:824] T 00000000000000000000000000000000 P 8702d46265dd4ab7be55d021e7de88b0: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:47.688433 6130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8702d46265dd4ab7be55d021e7de88b0: No bootstrap required, opened a new log
I20260808 02:18:47.692888 6130 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8702d46265dd4ab7be55d021e7de88b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8702d46265dd4ab7be55d021e7de88b0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 42389 } }
I20260808 02:18:47.693450 6130 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8702d46265dd4ab7be55d021e7de88b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:47.693759 6130 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8702d46265dd4ab7be55d021e7de88b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8702d46265dd4ab7be55d021e7de88b0, State: Initialized, Role: FOLLOWER
I20260808 02:18:47.694427 6130 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8702d46265dd4ab7be55d021e7de88b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8702d46265dd4ab7be55d021e7de88b0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 42389 } }
I20260808 02:18:47.694737 6130 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8702d46265dd4ab7be55d021e7de88b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:18:47.694849 6130 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8702d46265dd4ab7be55d021e7de88b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:18:47.694991 6130 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8702d46265dd4ab7be55d021e7de88b0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:47.696408 6130 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8702d46265dd4ab7be55d021e7de88b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8702d46265dd4ab7be55d021e7de88b0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 42389 } }
I20260808 02:18:47.696880 6130 leader_election.cc:302] T 00000000000000000000000000000000 P 8702d46265dd4ab7be55d021e7de88b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8702d46265dd4ab7be55d021e7de88b0; no voters:
I20260808 02:18:47.697388 6130 leader_election.cc:288] T 00000000000000000000000000000000 P 8702d46265dd4ab7be55d021e7de88b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:18:47.697526 6140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8702d46265dd4ab7be55d021e7de88b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:47.697844 6140 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8702d46265dd4ab7be55d021e7de88b0 [term 1 LEADER]: Becoming Leader. State: Replica: 8702d46265dd4ab7be55d021e7de88b0, State: Running, Role: LEADER
I20260808 02:18:47.698235 6140 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8702d46265dd4ab7be55d021e7de88b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8702d46265dd4ab7be55d021e7de88b0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 42389 } }
I20260808 02:18:47.699008 6130 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8702d46265dd4ab7be55d021e7de88b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:18:47.700824 6142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8702d46265dd4ab7be55d021e7de88b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8702d46265dd4ab7be55d021e7de88b0. Latest consensus state: current_term: 1 leader_uuid: "8702d46265dd4ab7be55d021e7de88b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8702d46265dd4ab7be55d021e7de88b0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 42389 } } }
I20260808 02:18:47.700976 6142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8702d46265dd4ab7be55d021e7de88b0 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:47.700788 6141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8702d46265dd4ab7be55d021e7de88b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8702d46265dd4ab7be55d021e7de88b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8702d46265dd4ab7be55d021e7de88b0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 42389 } } }
I20260808 02:18:47.701378 6141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8702d46265dd4ab7be55d021e7de88b0 [sys.catalog]: This master's current role is: LEADER
W20260808 02:18:47.704479 6159 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8702d46265dd4ab7be55d021e7de88b0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:18:47.704594 6159 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:18:47.704692 6158 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:18:47.705569 6158 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:18:47.713439 6158 catalog_manager.cc:1347] Generated new cluster ID: a3da1d7208b6454cb5358dd68f5be106
I20260808 02:18:47.713565 6158 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:18:47.724484 6158 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:18:47.725567 6158 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:18:47.735316 6158 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8702d46265dd4ab7be55d021e7de88b0: Generated new TSK 0
I20260808 02:18:47.736230 6158 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:18:47.745620 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:42389
--builtin_ntp_servers=127.24.79.212:33149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 02:18:47.877385 6165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:47.877753 6165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:47.877796 6165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:47.882238 6165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:47.882340 6165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:47.882400 6165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:18:47.887692 6165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:33149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:42389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:47.889014 6165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:47.890573 6165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:47.903553 6177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:47.903463 6165 server_base.cc:1034] running on GCE node
W20260808 02:18:47.903659 6173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:47.903697 6175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:47.904395 6165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:47.905200 6165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:47.906420 6165 hybrid_clock.cc:648] HybridClock initialized: now 1786155527906377 us; error 57 us; skew 500 ppm
I20260808 02:18:47.909287 6165 webserver.cc:460] Webserver started at http://127.24.79.193:46309/ using document root <none> and password file <none>
I20260808 02:18:47.910029 6165 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:47.910157 6165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:47.910384 6165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:47.913223 6165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "188996e76584463ca283062576e5db18"
format_stamp: "Formatted at 2026-08-08 02:18:47 on dist-test-slave-1vmg"
server_key: "75fd36ce17d84fd30a9ef8c06381c5d4"
server_key_iv: "3cf382eb0d1c490b7e070b8a09cac133"
server_key_version: "encryptionkey@0"
I20260808 02:18:47.913893 6165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "188996e76584463ca283062576e5db18"
format_stamp: "Formatted at 2026-08-08 02:18:47 on dist-test-slave-1vmg"
server_key: "75fd36ce17d84fd30a9ef8c06381c5d4"
server_key_iv: "3cf382eb0d1c490b7e070b8a09cac133"
server_key_version: "encryptionkey@0"
I20260808 02:18:47.918563 6165 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:18:47.922358 6185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:47.924280 6165 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:47.924480 6165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "188996e76584463ca283062576e5db18"
format_stamp: "Formatted at 2026-08-08 02:18:47 on dist-test-slave-1vmg"
server_key: "75fd36ce17d84fd30a9ef8c06381c5d4"
server_key_iv: "3cf382eb0d1c490b7e070b8a09cac133"
server_key_version: "encryptionkey@0"
I20260808 02:18:47.924634 6165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:47.968986 6165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:47.969913 6165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:47.970132 6165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:47.970844 6165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:47.972039 6165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:47.972096 6165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:47.972158 6165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:47.972188 6165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:47.981858 6165 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:41183
I20260808 02:18:47.981943 6349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:41183 every 8 connection(s)
I20260808 02:18:47.982955 6165 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:18:47.985988 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 6165
I20260808 02:18:47.986253 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:18:47.986852 24895 external_mini_cluster.cc:1428] Setting key 5fd71ce43df265f920b4d2ea49abeffe
I20260808 02:18:47.989264 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:42389
--builtin_ntp_servers=127.24.79.212:33149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 02:18:47.995002 6138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:47.985270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:59487 (local address 127.24.79.254:42389)
0808 02:18:47.985463 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:47.985468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:47.987244 (+ 1776us) server_negotiation.cc:407] Connection header received
0808 02:18:47.988948 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:47.988952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:47.989024 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:47.989105 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:47.990320 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:47.990882 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:47.992265 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:47.992505 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:47.992614 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:47.994350 (+ 1736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:47.994354 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:47.994356 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:18:47.994375 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:47.994400 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:18:47.994409 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:47.994481 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:47.994681 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 02:18:47.994796 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 02:18:47.996090 6353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:47.985823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:42389 (local address 127.24.79.193:59487)
0808 02:18:47.986986 (+ 1163us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:47.987059 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:47.988695 (+ 1636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:47.989360 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:47.989371 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:47.989517 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:47.990125 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:47.990154 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:47.991065 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:47.991069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:47.992088 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:47.992103 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:47.992719 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:47.992724 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:47.992928 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:47.993882 (+ 954us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:18:47.993884 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:47.993949 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:18:47.994140 (+ 191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:47.994499 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:47.994513 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:47.994521 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 02:18:47.994659 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 02:18:47.995101 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":320,"threads_started":1}
I20260808 02:18:47.997997 6351 heartbeater.cc:347] Connected to a master server at 127.24.79.254:42389
I20260808 02:18:47.998332 6351 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:47.999145 6351 heartbeater.cc:511] Master 127.24.79.254:42389 requested a full tablet report, sending...
I20260808 02:18:48.001591 6071 ts_manager.cc:194] Registered new tserver with Master: 188996e76584463ca283062576e5db18 (127.24.79.193:41183)
I20260808 02:18:48.002907 6071 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.79.193:59487
W20260808 02:18:48.126286 6354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:48.126659 6354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:48.126719 6354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:48.131593 6354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:48.131690 6354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:48.131776 6354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:18:48.137449 6354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:33149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:42389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:48.138834 6354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:48.140587 6354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:48.152431 6360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:48.152472 6363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:48.152472 6361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:48.152771 6354 server_base.cc:1034] running on GCE node
I20260808 02:18:48.153380 6354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:48.154187 6354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:48.155390 6354 hybrid_clock.cc:648] HybridClock initialized: now 1786155528155353 us; error 52 us; skew 500 ppm
I20260808 02:18:48.157781 6354 webserver.cc:460] Webserver started at http://127.24.79.194:43281/ using document root <none> and password file <none>
I20260808 02:18:48.158449 6354 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:48.158521 6354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:48.158763 6354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:48.161465 6354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "c7218245cfe64834be33395f47befb87"
format_stamp: "Formatted at 2026-08-08 02:18:48 on dist-test-slave-1vmg"
server_key: "cab04e6e9acc5c2ea136d535d812211d"
server_key_iv: "f0ddbbb1ca3c079e6569634684883170"
server_key_version: "encryptionkey@0"
I20260808 02:18:48.162230 6354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "c7218245cfe64834be33395f47befb87"
format_stamp: "Formatted at 2026-08-08 02:18:48 on dist-test-slave-1vmg"
server_key: "cab04e6e9acc5c2ea136d535d812211d"
server_key_iv: "f0ddbbb1ca3c079e6569634684883170"
server_key_version: "encryptionkey@0"
I20260808 02:18:48.167219 6354 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 02:18:48.170567 6374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:48.172394 6354 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:48.172609 6354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "c7218245cfe64834be33395f47befb87"
format_stamp: "Formatted at 2026-08-08 02:18:48 on dist-test-slave-1vmg"
server_key: "cab04e6e9acc5c2ea136d535d812211d"
server_key_iv: "f0ddbbb1ca3c079e6569634684883170"
server_key_version: "encryptionkey@0"
I20260808 02:18:48.172770 6354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:48.187367 6354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:48.188263 6354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:48.188482 6354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:48.189199 6354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:48.190382 6354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:48.190441 6354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:48.190619 6354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:48.190657 6354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:48.201954 6354 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:33099
I20260808 02:18:48.201970 6528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:33099 every 8 connection(s)
I20260808 02:18:48.203179 6354 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:18:48.209488 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 6354
I20260808 02:18:48.209612 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:18:48.209945 24895 external_mini_cluster.cc:1428] Setting key e09a6444b0e676048b1cff1ff2380b37
I20260808 02:18:48.212553 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:42389
--builtin_ntp_servers=127.24.79.212:33149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 02:18:48.216253 6138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:48.205439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:58701 (local address 127.24.79.254:42389)
0808 02:18:48.205630 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:48.205635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:48.207356 (+ 1721us) server_negotiation.cc:407] Connection header received
0808 02:18:48.209086 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:48.209089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:48.209211 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:48.209341 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:48.210589 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:48.211329 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:48.213540 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:48.213797 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:48.213905 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:48.215544 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:48.215549 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:48.215551 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:18:48.215571 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:48.215643 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:18:48.215656 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:48.215754 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:48.215990 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 02:18:48.216081 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 02:18:48.217329 6532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:48.206159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:42389 (local address 127.24.79.194:58701)
0808 02:18:48.207130 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:48.207197 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:48.208772 (+ 1575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:48.209602 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:48.209612 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:48.209750 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:48.210399 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:48.210416 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:48.211624 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:48.211628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:48.213343 (+ 1715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:48.213361 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:48.213997 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:48.214001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:48.214247 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:48.215167 (+ 920us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:18:48.215169 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:48.215224 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:18:48.215316 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:48.215791 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:48.215808 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:48.215815 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 02:18:48.215962 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 02:18:48.216328 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":183,"threads_started":1}
I20260808 02:18:48.218863 6529 heartbeater.cc:347] Connected to a master server at 127.24.79.254:42389
I20260808 02:18:48.219234 6529 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:48.220075 6529 heartbeater.cc:511] Master 127.24.79.254:42389 requested a full tablet report, sending...
I20260808 02:18:48.221526 6071 ts_manager.cc:194] Registered new tserver with Master: c7218245cfe64834be33395f47befb87 (127.24.79.194:33099)
I20260808 02:18:48.222370 6071 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.79.194:58701
W20260808 02:18:48.344074 6534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:48.344416 6534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:48.344456 6534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:48.348841 6534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:48.348928 6534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:48.348987 6534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:18:48.354317 6534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:33149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:42389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:48.355791 6534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:48.357785 6534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:48.369748 6543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:48.369815 6540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:48.369834 6539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:48.369938 6534 server_base.cc:1034] running on GCE node
I20260808 02:18:48.370551 6534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:48.371428 6534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:48.372665 6534 hybrid_clock.cc:648] HybridClock initialized: now 1786155528372645 us; error 36 us; skew 500 ppm
I20260808 02:18:48.375115 6534 webserver.cc:460] Webserver started at http://127.24.79.195:39603/ using document root <none> and password file <none>
I20260808 02:18:48.375770 6534 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:48.375838 6534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:48.376026 6534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:48.378404 6534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "a90abf34a6784f828bfb2cf12c22a471"
format_stamp: "Formatted at 2026-08-08 02:18:48 on dist-test-slave-1vmg"
server_key: "4478835f85326df87514bdb42f483462"
server_key_iv: "b58ca805c2e723f238c5b9691a368822"
server_key_version: "encryptionkey@0"
I20260808 02:18:48.379014 6534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "a90abf34a6784f828bfb2cf12c22a471"
format_stamp: "Formatted at 2026-08-08 02:18:48 on dist-test-slave-1vmg"
server_key: "4478835f85326df87514bdb42f483462"
server_key_iv: "b58ca805c2e723f238c5b9691a368822"
server_key_version: "encryptionkey@0"
I20260808 02:18:48.383414 6534 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:18:48.386545 6553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:48.388407 6534 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:48.388561 6534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "a90abf34a6784f828bfb2cf12c22a471"
format_stamp: "Formatted at 2026-08-08 02:18:48 on dist-test-slave-1vmg"
server_key: "4478835f85326df87514bdb42f483462"
server_key_iv: "b58ca805c2e723f238c5b9691a368822"
server_key_version: "encryptionkey@0"
I20260808 02:18:48.388691 6534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:48.414477 6534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:48.415441 6534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:48.415643 6534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:48.416298 6534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:48.417445 6534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:48.417511 6534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:48.417555 6534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:48.417569 6534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:48.427623 6534 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:38007
I20260808 02:18:48.427699 6720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:38007 every 8 connection(s)
I20260808 02:18:48.428725 6534 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:18:48.432830 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 6534
I20260808 02:18:48.432946 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:18:48.433248 24895 external_mini_cluster.cc:1428] Setting key 6e52a975af1847d25f3e979e05621e48
I20260808 02:18:48.439461 6138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:48.431040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:59997 (local address 127.24.79.254:42389)
0808 02:18:48.431263 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:48.431267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:48.432527 (+ 1260us) server_negotiation.cc:407] Connection header received
0808 02:18:48.434118 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:48.434121 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:48.434194 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:48.434292 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:48.435654 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:48.436189 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:48.437375 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:48.437599 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:48.437688 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:48.438890 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:48.438893 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:48.438896 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:18:48.438941 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:48.438966 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:18:48.438977 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:48.439046 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:48.439230 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 02:18:48.439305 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 02:18:48.440447 6726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:48.431508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:42389 (local address 127.24.79.195:59997)
0808 02:18:48.432321 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:48.432370 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:48.433904 (+ 1534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:48.434517 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:48.434525 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:48.434621 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:48.435504 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:48.435528 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:48.436325 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:48.436327 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:48.437251 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:48.437261 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:48.437760 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:48.437765 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:48.437935 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:48.438597 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:18:48.438599 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:48.438643 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:18:48.438728 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:48.439072 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:48.439083 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:48.439089 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 02:18:48.439217 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 02:18:48.439549 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":150,"threads_started":1}
I20260808 02:18:48.441825 6723 heartbeater.cc:347] Connected to a master server at 127.24.79.254:42389
I20260808 02:18:48.442119 6723 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:48.442783 6723 heartbeater.cc:511] Master 127.24.79.254:42389 requested a full tablet report, sending...
I20260808 02:18:48.444053 6071 ts_manager.cc:194] Registered new tserver with Master: a90abf34a6784f828bfb2cf12c22a471 (127.24.79.195:38007)
I20260808 02:18:48.444692 6071 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.79.195:59997
I20260808 02:18:48.447795 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 02:18:48.450994 6138 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:48.449892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44976 (local address 127.24.79.254:42389)
0808 02:18:48.450027 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:48.450031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:48.450251 (+ 220us) server_negotiation.cc:407] Connection header received
0808 02:18:48.450387 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:48.450391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:48.450437 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:48.450504 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:48.450809 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44976: BlockingRecv error: recv got EOF from 127.0.0.1:44976 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 02:18:48.451503 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 6165
I20260808 02:18:48.465301 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 6354
I20260808 02:18:48.485466 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 6534
I20260808 02:18:48.506544 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 6028
2026-08-08T02:18:48Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (1088 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T02:18:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:18:48Z Disabled control of system clock
I20260808 02:18:48.549018 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:34173
--webserver_interface=127.24.79.254
--webserver_port=0
--builtin_ntp_servers=127.24.79.212:33313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:34173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 02:18:48.673959 6737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:48.674273 6737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:48.674312 6737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:48.678550 6737 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:18:48.678654 6737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:48.678669 6737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:48.678686 6737 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:18:48.678702 6737 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:18:48.683902 6737 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:33313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:34173
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:34173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:48.685197 6737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:48.686825 6737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:48.696520 6752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:48.696539 6749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:48.696520 6748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:48.696990 6737 server_base.cc:1034] running on GCE node
I20260808 02:18:48.699004 6737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:48.700903 6737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:48.702189 6737 hybrid_clock.cc:648] HybridClock initialized: now 1786155528702162 us; error 46 us; skew 500 ppm
I20260808 02:18:48.704468 6737 webserver.cc:460] Webserver started at http://127.24.79.254:33041/ using document root <none> and password file <none>
I20260808 02:18:48.705070 6737 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:48.705148 6737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:48.705338 6737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:48.707753 6737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "54c052e9909d4ee192df9efa74a07868"
format_stamp: "Formatted at 2026-08-08 02:18:48 on dist-test-slave-1vmg"
server_key: "0c2a4496cb8c43454d04b7441a584327"
server_key_iv: "814e0f8a5cb2d205656aaf001c89a2af"
server_key_version: "encryptionkey@0"
I20260808 02:18:48.708357 6737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "54c052e9909d4ee192df9efa74a07868"
format_stamp: "Formatted at 2026-08-08 02:18:48 on dist-test-slave-1vmg"
server_key: "0c2a4496cb8c43454d04b7441a584327"
server_key_iv: "814e0f8a5cb2d205656aaf001c89a2af"
server_key_version: "encryptionkey@0"
I20260808 02:18:48.712968 6737 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:18:48.715932 6762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:48.717464 6737 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:48.717608 6737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "54c052e9909d4ee192df9efa74a07868"
format_stamp: "Formatted at 2026-08-08 02:18:48 on dist-test-slave-1vmg"
server_key: "0c2a4496cb8c43454d04b7441a584327"
server_key_iv: "814e0f8a5cb2d205656aaf001c89a2af"
server_key_version: "encryptionkey@0"
I20260808 02:18:48.717732 6737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:48.723088 6737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:48.723804 6737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:48.723973 6737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:48.732062 6737 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:34173
I20260808 02:18:48.732080 6841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:34173 every 8 connection(s)
I20260808 02:18:48.733392 6737 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:18:48.737162 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 6737
I20260808 02:18:48.737296 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:18:48.737488 6842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:48.737629 24895 external_mini_cluster.cc:1428] Setting key 26006ebce1a6696f672e9d6e3072690d
I20260808 02:18:48.744580 6847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:48.739291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33056 (local address 127.24.79.254:34173)
0808 02:18:48.740100 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:48.740121 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:48.740166 (+ 45us) server_negotiation.cc:407] Connection header received
0808 02:18:48.741126 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:48.741151 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:48.741580 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:48.741928 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:48.743128 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:48.743136 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:48.743151 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:18:48.743167 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:48.743244 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:18:48.743265 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:48.743415 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:48.743519 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 02:18:48.743803 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":147,"threads_started":1}
I20260808 02:18:48.745428 6842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c052e9909d4ee192df9efa74a07868: Bootstrap starting.
I20260808 02:18:48.747965 6842 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54c052e9909d4ee192df9efa74a07868: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:48.748799 6842 log.cc:824] T 00000000000000000000000000000000 P 54c052e9909d4ee192df9efa74a07868: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:48.751453 6842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c052e9909d4ee192df9efa74a07868: No bootstrap required, opened a new log
I20260808 02:18:48.755388 6842 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54c052e9909d4ee192df9efa74a07868 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c052e9909d4ee192df9efa74a07868" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 34173 } }
I20260808 02:18:48.755697 6842 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54c052e9909d4ee192df9efa74a07868 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:48.755753 6842 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54c052e9909d4ee192df9efa74a07868 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54c052e9909d4ee192df9efa74a07868, State: Initialized, Role: FOLLOWER
I20260808 02:18:48.756258 6842 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54c052e9909d4ee192df9efa74a07868 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c052e9909d4ee192df9efa74a07868" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 34173 } }
I20260808 02:18:48.756402 6842 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54c052e9909d4ee192df9efa74a07868 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:18:48.756462 6842 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54c052e9909d4ee192df9efa74a07868 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:18:48.756573 6842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54c052e9909d4ee192df9efa74a07868 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:48.757645 6842 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54c052e9909d4ee192df9efa74a07868 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c052e9909d4ee192df9efa74a07868" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 34173 } }
I20260808 02:18:48.758044 6842 leader_election.cc:302] T 00000000000000000000000000000000 P 54c052e9909d4ee192df9efa74a07868 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54c052e9909d4ee192df9efa74a07868; no voters:
I20260808 02:18:48.758417 6842 leader_election.cc:288] T 00000000000000000000000000000000 P 54c052e9909d4ee192df9efa74a07868 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:18:48.758642 6849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54c052e9909d4ee192df9efa74a07868 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:48.758906 6849 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54c052e9909d4ee192df9efa74a07868 [term 1 LEADER]: Becoming Leader. State: Replica: 54c052e9909d4ee192df9efa74a07868, State: Running, Role: LEADER
I20260808 02:18:48.759599 6849 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54c052e9909d4ee192df9efa74a07868 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c052e9909d4ee192df9efa74a07868" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 34173 } }
I20260808 02:18:48.760010 6842 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54c052e9909d4ee192df9efa74a07868 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:18:48.761447 6851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c052e9909d4ee192df9efa74a07868 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54c052e9909d4ee192df9efa74a07868" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c052e9909d4ee192df9efa74a07868" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 34173 } } }
I20260808 02:18:48.761447 6852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c052e9909d4ee192df9efa74a07868 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54c052e9909d4ee192df9efa74a07868. Latest consensus state: current_term: 1 leader_uuid: "54c052e9909d4ee192df9efa74a07868" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c052e9909d4ee192df9efa74a07868" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 34173 } } }
I20260808 02:18:48.761623 6851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c052e9909d4ee192df9efa74a07868 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:48.761623 6852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c052e9909d4ee192df9efa74a07868 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:48.762238 6862 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:18:48.765892 6862 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:18:48.772835 6862 catalog_manager.cc:1347] Generated new cluster ID: 0bc034560df74aafbf9f3dee28eb1a01
I20260808 02:18:48.772935 6862 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:18:48.796344 6862 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:18:48.796783 6862 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:18:48.805301 6862 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54c052e9909d4ee192df9efa74a07868: Generated new TSK 0
I20260808 02:18:48.806150 6862 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:18:48.818295 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:34173
--builtin_ntp_servers=127.24.79.212:33313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 02:18:48.948313 6872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:48.948657 6872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:48.948704 6872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:48.952988 6872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:48.953092 6872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:48.953154 6872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:18:48.958372 6872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:33313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:34173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:48.959738 6872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:48.961462 6872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:48.972532 6883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:48.972550 6879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:48.972630 6872 server_base.cc:1034] running on GCE node
W20260808 02:18:48.972546 6880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:48.973291 6872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:48.974030 6872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:48.975221 6872 hybrid_clock.cc:648] HybridClock initialized: now 1786155528975185 us; error 49 us; skew 500 ppm
I20260808 02:18:48.977382 6872 webserver.cc:460] Webserver started at http://127.24.79.193:33457/ using document root <none> and password file <none>
I20260808 02:18:48.978026 6872 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:48.978109 6872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:48.978338 6872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:48.980757 6872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "db7ee1f93b5c46a4a369fbaa37bf60ce"
format_stamp: "Formatted at 2026-08-08 02:18:48 on dist-test-slave-1vmg"
server_key: "607d22bd0d42628b4d55ea214ffa9207"
server_key_iv: "36be5d19f9928c9392fc0b80b2c23f2d"
server_key_version: "encryptionkey@0"
I20260808 02:18:48.981431 6872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "db7ee1f93b5c46a4a369fbaa37bf60ce"
format_stamp: "Formatted at 2026-08-08 02:18:48 on dist-test-slave-1vmg"
server_key: "607d22bd0d42628b4d55ea214ffa9207"
server_key_iv: "36be5d19f9928c9392fc0b80b2c23f2d"
server_key_version: "encryptionkey@0"
I20260808 02:18:48.986023 6872 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:18:48.989857 6892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:48.991410 6872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:48.991544 6872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "db7ee1f93b5c46a4a369fbaa37bf60ce"
format_stamp: "Formatted at 2026-08-08 02:18:48 on dist-test-slave-1vmg"
server_key: "607d22bd0d42628b4d55ea214ffa9207"
server_key_iv: "36be5d19f9928c9392fc0b80b2c23f2d"
server_key_version: "encryptionkey@0"
I20260808 02:18:48.991675 6872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:48.996984 6872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:48.997835 6872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:48.998059 6872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:48.998757 6872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:48.999991 6872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:49.000064 6872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:49.000131 6872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:49.000162 6872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:49.009995 6872 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:46761
I20260808 02:18:49.010073 7052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:46761 every 8 connection(s)
I20260808 02:18:49.011246 6872 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:18:49.017371 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 6872
I20260808 02:18:49.017495 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:18:49.017814 24895 external_mini_cluster.cc:1428] Setting key 4a570897276848a1677fc00b65d0b82d
I20260808 02:18:49.018898 6847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:49.013651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:59565 (local address 127.24.79.254:34173)
0808 02:18:49.013841 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:49.013845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:49.015219 (+ 1374us) server_negotiation.cc:407] Connection header received
0808 02:18:49.016717 (+ 1498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:49.016721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:49.016780 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:49.016866 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:49.018404 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:49.018409 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:49.018412 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:18:49.018429 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:49.018457 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:18:49.018468 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:49.018569 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:49.018736 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 02:18:49.018782 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 02:18:49.020144 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:49.014082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:34173 (local address 127.24.79.193:59565)
0808 02:18:49.015027 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:49.015091 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:49.016433 (+ 1342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:49.017097 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:49.017106 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:49.017342 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:49.018040 (+ 698us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:18:49.018042 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:49.018093 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:18:49.018183 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:49.018576 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:49.018585 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:49.018591 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 02:18:49.018796 (+ 205us) client_negotiation.cc:240] Negotiation successful
0808 02:18:49.019129 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":159,"threads_started":1}
I20260808 02:18:49.020164 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:34173
--builtin_ntp_servers=127.24.79.212:33313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 02:18:49.021587 7053 heartbeater.cc:347] Connected to a master server at 127.24.79.254:34173
I20260808 02:18:49.021894 7053 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:49.022503 7053 heartbeater.cc:511] Master 127.24.79.254:34173 requested a full tablet report, sending...
I20260808 02:18:49.024457 6791 ts_manager.cc:194] Registered new tserver with Master: db7ee1f93b5c46a4a369fbaa37bf60ce (127.24.79.193:46761)
W20260808 02:18:49.146894 7056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:49.147308 7056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:49.147375 7056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:49.151903 7056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:49.151999 7056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:49.152083 7056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:18:49.157274 7056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:33313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:34173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:49.158689 7056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:49.160415 7056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:49.172075 7063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:49.172166 7064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:49.172392 7056 server_base.cc:1034] running on GCE node
W20260808 02:18:49.172082 7066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:49.172955 7056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:49.173702 7056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:49.174883 7056 hybrid_clock.cc:648] HybridClock initialized: now 1786155529174857 us; error 37 us; skew 500 ppm
I20260808 02:18:49.176972 7056 webserver.cc:460] Webserver started at http://127.24.79.194:46681/ using document root <none> and password file <none>
I20260808 02:18:49.177588 7056 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:49.177659 7056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:49.177847 7056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:49.180186 7056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "96d51ae1715e454984e9228ec246d888"
format_stamp: "Formatted at 2026-08-08 02:18:49 on dist-test-slave-1vmg"
server_key: "84b73952a13cb045ccdc9d70efc0a35c"
server_key_iv: "ff0523882b6b33458e52019387819fdb"
server_key_version: "encryptionkey@0"
I20260808 02:18:49.180763 7056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "96d51ae1715e454984e9228ec246d888"
format_stamp: "Formatted at 2026-08-08 02:18:49 on dist-test-slave-1vmg"
server_key: "84b73952a13cb045ccdc9d70efc0a35c"
server_key_iv: "ff0523882b6b33458e52019387819fdb"
server_key_version: "encryptionkey@0"
I20260808 02:18:49.185001 7056 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:18:49.188375 7073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:49.189864 7056 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:18:49.190017 7056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "96d51ae1715e454984e9228ec246d888"
format_stamp: "Formatted at 2026-08-08 02:18:49 on dist-test-slave-1vmg"
server_key: "84b73952a13cb045ccdc9d70efc0a35c"
server_key_iv: "ff0523882b6b33458e52019387819fdb"
server_key_version: "encryptionkey@0"
I20260808 02:18:49.190171 7056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:49.195472 7056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:49.196254 7056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:49.196468 7056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:49.197113 7056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:49.198227 7056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:49.198287 7056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:49.198347 7056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:49.198375 7056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:49.207970 7056 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:46303
I20260808 02:18:49.208019 7247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:46303 every 8 connection(s)
I20260808 02:18:49.208997 7056 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:18:49.215765 6847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:49.211137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:58225 (local address 127.24.79.254:34173)
0808 02:18:49.211313 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:49.211317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:49.212487 (+ 1170us) server_negotiation.cc:407] Connection header received
0808 02:18:49.213856 (+ 1369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:49.213858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:49.213905 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:49.213987 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:49.215259 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:49.215263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:49.215266 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:18:49.215281 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:49.215306 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:18:49.215316 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:49.215409 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:49.215589 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 02:18:49.215640 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:18:49.216687 7252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:49.211560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:34173 (local address 127.24.79.194:58225)
0808 02:18:49.212317 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:49.212363 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:49.213627 (+ 1264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:49.214164 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:49.214172 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:49.214308 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:49.214955 (+ 647us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:18:49.214957 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:49.215008 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:18:49.215089 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:49.215408 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:49.215415 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:49.215421 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 02:18:49.215564 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 02:18:49.215786 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":121,"threads_started":1}
I20260808 02:18:49.218014 7248 heartbeater.cc:347] Connected to a master server at 127.24.79.254:34173
I20260808 02:18:49.218261 7248 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:49.218703 7248 heartbeater.cc:511] Master 127.24.79.254:34173 requested a full tablet report, sending...
I20260808 02:18:49.219213 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 7056
I20260808 02:18:49.219344 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:18:49.219647 24895 external_mini_cluster.cc:1428] Setting key ae9d13788b169a6fe6f6b75ac5ea8976
I20260808 02:18:49.220096 6791 ts_manager.cc:194] Registered new tserver with Master: 96d51ae1715e454984e9228ec246d888 (127.24.79.194:46303)
I20260808 02:18:49.222110 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:34173
--builtin_ntp_servers=127.24.79.212:33313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 02:18:49.346150 7253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:49.346470 7253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:49.346511 7253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:49.350704 7253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:49.350790 7253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:49.350853 7253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:18:49.355957 7253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:33313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:34173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:49.357260 7253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:49.358800 7253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:49.369529 7265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:49.369532 7259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:49.369534 7260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:49.370640 7253 server_base.cc:1034] running on GCE node
I20260808 02:18:49.371294 7253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:49.372038 7253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:49.373258 7253 hybrid_clock.cc:648] HybridClock initialized: now 1786155529373242 us; error 32 us; skew 500 ppm
I20260808 02:18:49.375489 7253 webserver.cc:460] Webserver started at http://127.24.79.195:38145/ using document root <none> and password file <none>
I20260808 02:18:49.376116 7253 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:49.376204 7253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:49.376436 7253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:49.378674 7253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "16c47f609a664d95b4854bbafbf85091"
format_stamp: "Formatted at 2026-08-08 02:18:49 on dist-test-slave-1vmg"
server_key: "96d96e391c474488ae93d118e0b1e4ed"
server_key_iv: "0d5fc2be2e36c8f8c9f88e632f82e0de"
server_key_version: "encryptionkey@0"
I20260808 02:18:49.379336 7253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "16c47f609a664d95b4854bbafbf85091"
format_stamp: "Formatted at 2026-08-08 02:18:49 on dist-test-slave-1vmg"
server_key: "96d96e391c474488ae93d118e0b1e4ed"
server_key_iv: "0d5fc2be2e36c8f8c9f88e632f82e0de"
server_key_version: "encryptionkey@0"
I20260808 02:18:49.383621 7253 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:18:49.386636 7274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:49.388322 7253 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:49.388482 7253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "16c47f609a664d95b4854bbafbf85091"
format_stamp: "Formatted at 2026-08-08 02:18:49 on dist-test-slave-1vmg"
server_key: "96d96e391c474488ae93d118e0b1e4ed"
server_key_iv: "0d5fc2be2e36c8f8c9f88e632f82e0de"
server_key_version: "encryptionkey@0"
I20260808 02:18:49.388629 7253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:49.393760 7253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:49.394459 7253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:49.394646 7253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:49.395340 7253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:49.396409 7253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:49.396461 7253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:49.396514 7253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:49.396543 7253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:49.406010 7253 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:36305
I20260808 02:18:49.406047 7433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:36305 every 8 connection(s)
I20260808 02:18:49.407109 7253 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:18:49.410470 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 7253
I20260808 02:18:49.410598 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:18:49.410984 24895 external_mini_cluster.cc:1428] Setting key bcf34413366d6ea284b9fb32ca9bcec7
I20260808 02:18:49.414426 6847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:49.409316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:36335 (local address 127.24.79.254:34173)
0808 02:18:49.409486 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:49.409491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:49.410897 (+ 1406us) server_negotiation.cc:407] Connection header received
0808 02:18:49.412684 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:49.412687 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:49.412733 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:49.412817 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:49.414035 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:49.414038 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:49.414040 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:18:49.414053 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:49.414074 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:18:49.414083 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:49.414153 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:49.414276 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 02:18:49.414314 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 02:18:49.415503 7439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:49.409738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:34173 (local address 127.24.79.195:36335)
0808 02:18:49.410662 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:49.410727 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:49.412435 (+ 1708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:49.412978 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:49.412986 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:49.413137 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:49.413758 (+ 621us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:18:49.413760 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:49.413804 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:18:49.413879 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:49.414151 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:49.414156 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:49.414160 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:18:49.414295 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 02:18:49.414524 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":116,"threads_started":1}
I20260808 02:18:49.416716 7435 heartbeater.cc:347] Connected to a master server at 127.24.79.254:34173
I20260808 02:18:49.416988 7435 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:49.417507 7435 heartbeater.cc:511] Master 127.24.79.254:34173 requested a full tablet report, sending...
I20260808 02:18:49.418756 6790 ts_manager.cc:194] Registered new tserver with Master: 16c47f609a664d95b4854bbafbf85091 (127.24.79.195:36305)
I20260808 02:18:49.425817 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 02:18:49.428941 6847 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:18:49.427947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33072 (local address 127.24.79.254:34173)
0808 02:18:49.428131 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:49.428136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:49.428295 (+ 159us) server_negotiation.cc:407] Connection header received
0808 02:18:49.428429 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:49.428433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:49.428471 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:49.428537 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:49.428822 (+ 285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33072: BlockingRecv error: recv got EOF from 127.0.0.1:33072 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 02:18:49.429487 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 6872
I20260808 02:18:49.452821 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 7056
I20260808 02:18:49.475819 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 7253
I20260808 02:18:49.486788 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 6737
2026-08-08T02:18:49Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (975 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:18:49 dist-test-slave-1vmg krb5kdc[7456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:18:49 dist-test-slave-1vmg krb5kdc[7456](info): set up 2 sockets
Aug 08 02:18:49 dist-test-slave-1vmg krb5kdc[7456](info): commencing operation
krb5kdc: starting...
W20260808 02:18:50.659960 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:18:50 dist-test-slave-1vmg krb5kdc[7456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:18:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:18:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:18:50.820091 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:45651
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:45651
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:50.945883 7474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:50.946223 7474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:50.946267 7474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:50.950548 7474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:18:50.950650 7474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:50.950670 7474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:50.950686 7474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:18:50.950701 7474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:18:50.956581 7474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:45651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:45651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:50.957875 7474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:50.959038 7474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:50.968166 7486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:50.968328 7479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:50.968412 7483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:50.968734 7474 server_base.cc:1034] running on GCE node
I20260808 02:18:50.969367 7474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:50.970587 7474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:50.971771 7474 hybrid_clock.cc:648] HybridClock initialized: now 1786155530971757 us; error 33 us; skew 500 ppm
Aug 08 02:18:50 dist-test-slave-1vmg krb5kdc[7456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155530, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:50.977950 7474 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:18:50.979218 7474 webserver.cc:460] Webserver started at http://127.24.79.254:33259/ using document root <none> and password file <none>
I20260808 02:18:50.979869 7474 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:50.979948 7474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:50.980194 7474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:50.982546 7474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "80bcb614f6af4d34acfa6874fee5dd03"
format_stamp: "Formatted at 2026-08-08 02:18:50 on dist-test-slave-1vmg"
server_key: "f6b6da42e4a72294516e2e676c302e25"
server_key_iv: "8ff24d10cb8b6fe1f6be148d955a543d"
server_key_version: "encryptionkey@0"
I20260808 02:18:50.983233 7474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "80bcb614f6af4d34acfa6874fee5dd03"
format_stamp: "Formatted at 2026-08-08 02:18:50 on dist-test-slave-1vmg"
server_key: "f6b6da42e4a72294516e2e676c302e25"
server_key_iv: "8ff24d10cb8b6fe1f6be148d955a543d"
server_key_version: "encryptionkey@0"
I20260808 02:18:50.987965 7474 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:18:50.991145 7499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:50.992584 7474 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:18:50.992745 7474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "80bcb614f6af4d34acfa6874fee5dd03"
format_stamp: "Formatted at 2026-08-08 02:18:50 on dist-test-slave-1vmg"
server_key: "f6b6da42e4a72294516e2e676c302e25"
server_key_iv: "8ff24d10cb8b6fe1f6be148d955a543d"
server_key_version: "encryptionkey@0"
I20260808 02:18:50.992874 7474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:51.006799 7474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:51.011152 7474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:51.011404 7474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:51.020072 7474 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:45651
I20260808 02:18:51.020095 7566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:45651 every 8 connection(s)
I20260808 02:18:51.021202 7474 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:18:51.024658 7567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:51.029875 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 7474
I20260808 02:18:51.030017 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:18:51.030304 24895 external_mini_cluster.cc:1428] Setting key dc9cf068ce8d08be7b44044d461a040f
I20260808 02:18:51.032023 7567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80bcb614f6af4d34acfa6874fee5dd03: Bootstrap starting.
I20260808 02:18:51.034842 7567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80bcb614f6af4d34acfa6874fee5dd03: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:51.035766 7567 log.cc:824] T 00000000000000000000000000000000 P 80bcb614f6af4d34acfa6874fee5dd03: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:51.038098 7567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80bcb614f6af4d34acfa6874fee5dd03: No bootstrap required, opened a new log
Aug 08 02:18:51 dist-test-slave-1vmg krb5kdc[7456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:18:51.042202 7567 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80bcb614f6af4d34acfa6874fee5dd03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80bcb614f6af4d34acfa6874fee5dd03" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45651 } }
I20260808 02:18:51.042531 7567 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80bcb614f6af4d34acfa6874fee5dd03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:51.042618 7567 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80bcb614f6af4d34acfa6874fee5dd03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80bcb614f6af4d34acfa6874fee5dd03, State: Initialized, Role: FOLLOWER
I20260808 02:18:51.043147 7567 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80bcb614f6af4d34acfa6874fee5dd03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80bcb614f6af4d34acfa6874fee5dd03" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45651 } }
I20260808 02:18:51.043294 7567 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80bcb614f6af4d34acfa6874fee5dd03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:18:51.043365 7567 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80bcb614f6af4d34acfa6874fee5dd03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:18:51.043496 7567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80bcb614f6af4d34acfa6874fee5dd03 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:51.044692 7567 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80bcb614f6af4d34acfa6874fee5dd03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80bcb614f6af4d34acfa6874fee5dd03" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45651 } }
I20260808 02:18:51.045090 7567 leader_election.cc:302] T 00000000000000000000000000000000 P 80bcb614f6af4d34acfa6874fee5dd03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80bcb614f6af4d34acfa6874fee5dd03; no voters:
I20260808 02:18:51.045511 7567 leader_election.cc:288] T 00000000000000000000000000000000 P 80bcb614f6af4d34acfa6874fee5dd03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:18:51.045655 7574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80bcb614f6af4d34acfa6874fee5dd03 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:51.045977 7574 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80bcb614f6af4d34acfa6874fee5dd03 [term 1 LEADER]: Becoming Leader. State: Replica: 80bcb614f6af4d34acfa6874fee5dd03, State: Running, Role: LEADER
I20260808 02:18:51.046391 7574 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80bcb614f6af4d34acfa6874fee5dd03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80bcb614f6af4d34acfa6874fee5dd03" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45651 } }
I20260808 02:18:51.047075 7567 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80bcb614f6af4d34acfa6874fee5dd03 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:18:51.047071 7572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:51.032199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47472 (local address 127.24.79.254:45651)
0808 02:18:51.032843 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:51.032864 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:51.032912 (+ 48us) server_negotiation.cc:407] Connection header received
0808 02:18:51.033896 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:51.033925 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:51.034412 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:51.034819 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:51.035852 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.036957 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.037895 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.038475 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.038642 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:51.041426 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:51.041431 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:51.041449 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:51.041483 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:51.043335 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:51.043885 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:51.043890 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:51.043895 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:51.044025 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:51.044341 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:51.044344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:51.044345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:51.044874 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:51.045059 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:51.045467 (+ 408us) server_negotiation.cc:299] Negotiation successful
0808 02:18:51.045860 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":159,"threads_started":1}
I20260808 02:18:51.048321 7575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80bcb614f6af4d34acfa6874fee5dd03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80bcb614f6af4d34acfa6874fee5dd03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80bcb614f6af4d34acfa6874fee5dd03" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45651 } } }
I20260808 02:18:51.048470 7575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80bcb614f6af4d34acfa6874fee5dd03 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:51.048913 7582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80bcb614f6af4d34acfa6874fee5dd03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80bcb614f6af4d34acfa6874fee5dd03. Latest consensus state: current_term: 1 leader_uuid: "80bcb614f6af4d34acfa6874fee5dd03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80bcb614f6af4d34acfa6874fee5dd03" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45651 } } }
I20260808 02:18:51.049077 7582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80bcb614f6af4d34acfa6874fee5dd03 [sys.catalog]: This master's current role is: LEADER
I20260808 02:18:51.049958 7595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:18:51.053062 7595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:18:51.059461 7595 catalog_manager.cc:1347] Generated new cluster ID: a4abffd2cec24112b100afe3fb77619a
I20260808 02:18:51.059569 7595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:18:51.076578 7595 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:18:51.077718 7595 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:18:51.084482 7595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80bcb614f6af4d34acfa6874fee5dd03: Generated new TSK 0
I20260808 02:18:51.085283 7595 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.79.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
I20260808 02:18:51.123301 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:45651
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:51.269392 7614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:51.269727 7614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:51.269774 7614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:51.274040 7614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:51.274127 7614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:51.274192 7614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:18:51.280012 7614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:45651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:51.281236 7614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:51.282320 7614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:51.293171 7621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:51.293146 7620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:51.293140 7630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:51.293597 7614 server_base.cc:1034] running on GCE node
I20260808 02:18:51.294076 7614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:51.294831 7614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:51.296015 7614 hybrid_clock.cc:648] HybridClock initialized: now 1786155531295984 us; error 42 us; skew 500 ppm
Aug 08 02:18:51 dist-test-slave-1vmg krb5kdc[7456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155531, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:51.299162 7614 init.cc:377] Logged in from keytab as kudu/127.24.79.193@KRBTEST.COM (short username kudu)
I20260808 02:18:51.300132 7614 webserver.cc:460] Webserver started at http://127.24.79.193:42865/ using document root <none> and password file <none>
I20260808 02:18:51.300705 7614 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:51.300768 7614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:51.300951 7614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:51.303146 7614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "9721a5cf82674929946d83d6b7dca403"
format_stamp: "Formatted at 2026-08-08 02:18:51 on dist-test-slave-1vmg"
server_key: "7ea00511a7610fa4384da9e5c1cae16f"
server_key_iv: "d54353e3d2ff40a0f169373e89c33b5a"
server_key_version: "encryptionkey@0"
I20260808 02:18:51.303733 7614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "9721a5cf82674929946d83d6b7dca403"
format_stamp: "Formatted at 2026-08-08 02:18:51 on dist-test-slave-1vmg"
server_key: "7ea00511a7610fa4384da9e5c1cae16f"
server_key_iv: "d54353e3d2ff40a0f169373e89c33b5a"
server_key_version: "encryptionkey@0"
I20260808 02:18:51.308007 7614 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:18:51.311060 7641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:51.312484 7614 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:18:51.312631 7614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "9721a5cf82674929946d83d6b7dca403"
format_stamp: "Formatted at 2026-08-08 02:18:51 on dist-test-slave-1vmg"
server_key: "7ea00511a7610fa4384da9e5c1cae16f"
server_key_iv: "d54353e3d2ff40a0f169373e89c33b5a"
server_key_version: "encryptionkey@0"
I20260808 02:18:51.312762 7614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:51.331262 7614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:51.335342 7614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:51.335575 7614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:51.336331 7614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:51.337337 7614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:51.337392 7614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:51.337453 7614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:51.337585 7614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:51.347374 7614 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:39113
I20260808 02:18:51.347402 7810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:39113 every 8 connection(s)
I20260808 02:18:51.348570 7614 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:18:51.354046 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 7614
I20260808 02:18:51.354205 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:18:51.354519 24895 external_mini_cluster.cc:1428] Setting key 548a2f3b8d4b258e126783cfebe0cb45
Aug 08 02:18:51 dist-test-slave-1vmg krb5kdc[7456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155531, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:18:51.365607 7572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:51.350794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:46445 (local address 127.24.79.254:45651)
0808 02:18:51.350969 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:51.350974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:51.351999 (+ 1025us) server_negotiation.cc:407] Connection header received
0808 02:18:51.353426 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:51.353430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:51.353490 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:51.353573 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:51.355289 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.356012 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.357680 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.357949 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.358058 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:51.361188 (+ 3130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:51.361192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:51.361195 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:51.361227 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:51.362771 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:51.363649 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:51.363654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:51.363655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:51.363752 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:51.364211 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:51.364214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:51.364215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:51.364425 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:51.364503 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:51.365331 (+ 828us) server_negotiation.cc:299] Negotiation successful
0808 02:18:51.365468 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
WARNING: no policy specified for kudu/127.24.79.194@KRBTEST.COM; defaulting to no policy
I20260808 02:18:51.366858 7816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:51.351272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45651 (local address 127.24.79.193:46445)
0808 02:18:51.351827 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:51.351877 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:51.353188 (+ 1311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:51.353781 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:51.353790 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:51.354319 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:51.355117 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:51.355143 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.356145 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.356148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:51.357496 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:51.357510 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.358134 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.358138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:51.358365 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:51.359099 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:51.359118 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:51.360908 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:51.362941 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:51.362950 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:51.362972 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:51.363414 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:51.363857 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:51.363861 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:51.363862 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:51.364081 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:51.364529 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:51.364537 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:51.364977 (+ 440us) client_negotiation.cc:769] Sending connection context
0808 02:18:51.365232 (+ 255us) client_negotiation.cc:240] Negotiation successful
0808 02:18:51.365652 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":132,"threads_started":1}
I20260808 02:18:51.368405 7813 heartbeater.cc:347] Connected to a master server at 127.24.79.254:45651
I20260808 02:18:51.368669 7813 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:51.369246 7813 heartbeater.cc:511] Master 127.24.79.254:45651 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.194@KRBTEST.COM" created.
I20260808 02:18:51.371145 7519 ts_manager.cc:194] Registered new tserver with Master: 9721a5cf82674929946d83d6b7dca403 (127.24.79.193:39113)
I20260808 02:18:51.372522 7519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.193@KRBTEST.COM'} at 127.24.79.193:46445
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
I20260808 02:18:51.386713 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:45651
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:51.508817 7819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:51.509155 7819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:51.509202 7819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:51.513484 7819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:51.513571 7819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:51.513638 7819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:18:51.519626 7819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:45651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:51.520896 7819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:51.522015 7819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:51.532968 7827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:51.532963 7826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:51.533227 7819 server_base.cc:1034] running on GCE node
W20260808 02:18:51.533088 7829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:51.533820 7819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:51.534686 7819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:51.535887 7819 hybrid_clock.cc:648] HybridClock initialized: now 1786155531535846 us; error 90 us; skew 500 ppm
Aug 08 02:18:51 dist-test-slave-1vmg krb5kdc[7456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155531, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:51.539084 7819 init.cc:377] Logged in from keytab as kudu/127.24.79.194@KRBTEST.COM (short username kudu)
I20260808 02:18:51.540367 7819 webserver.cc:460] Webserver started at http://127.24.79.194:42045/ using document root <none> and password file <none>
I20260808 02:18:51.540969 7819 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:51.541038 7819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:51.541239 7819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:51.543637 7819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "0f2234ed0df443649f0ad2b549a22782"
format_stamp: "Formatted at 2026-08-08 02:18:51 on dist-test-slave-1vmg"
server_key: "bbf28cceef1440d634dc3dd3d13ff7c8"
server_key_iv: "458c2d98421334cd367f6d5305de6be5"
server_key_version: "encryptionkey@0"
I20260808 02:18:51.544255 7819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "0f2234ed0df443649f0ad2b549a22782"
format_stamp: "Formatted at 2026-08-08 02:18:51 on dist-test-slave-1vmg"
server_key: "bbf28cceef1440d634dc3dd3d13ff7c8"
server_key_iv: "458c2d98421334cd367f6d5305de6be5"
server_key_version: "encryptionkey@0"
I20260808 02:18:51.548791 7819 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:18:51.551683 7840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:51.553196 7819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:18:51.553339 7819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "0f2234ed0df443649f0ad2b549a22782"
format_stamp: "Formatted at 2026-08-08 02:18:51 on dist-test-slave-1vmg"
server_key: "bbf28cceef1440d634dc3dd3d13ff7c8"
server_key_iv: "458c2d98421334cd367f6d5305de6be5"
server_key_version: "encryptionkey@0"
I20260808 02:18:51.553462 7819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:51.567857 7819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:51.572000 7819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:51.572227 7819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:51.572988 7819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:51.573968 7819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:51.574024 7819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:51.574082 7819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:51.574112 7819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:51.583693 7819 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:43473
I20260808 02:18:51.583798 8006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:43473 every 8 connection(s)
I20260808 02:18:51.584856 7819 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:18:51.585371 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 7819
I20260808 02:18:51.585493 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:18:51.585832 24895 external_mini_cluster.cc:1428] Setting key 91d8a6e4c53e6afc1ef617f9fb15dde2
Aug 08 02:18:51 dist-test-slave-1vmg krb5kdc[7456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155531, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.79.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.195@KRBTEST.COM" created.
I20260808 02:18:51.600994 7572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:51.587201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:52733 (local address 127.24.79.254:45651)
0808 02:18:51.587349 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:51.587354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:51.588489 (+ 1135us) server_negotiation.cc:407] Connection header received
0808 02:18:51.590003 (+ 1514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:51.590006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:51.590070 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:51.590158 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:51.591609 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.592161 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.593459 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.593703 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.593799 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:51.596853 (+ 3054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:51.596856 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:51.596859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:51.596891 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:51.598370 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:51.599012 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:51.599016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:51.599018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:51.599118 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:51.599596 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:51.599599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:51.599601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:51.599865 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:51.599986 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:51.600647 (+ 661us) server_negotiation.cc:299] Negotiation successful
0808 02:18:51.600829 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:18:51.601949 8011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:51.587740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45651 (local address 127.24.79.194:52733)
0808 02:18:51.588315 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:51.588365 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:51.589760 (+ 1395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:51.590373 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:51.590381 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:51.590812 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:51.591434 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:51.591451 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.592307 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.592310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:51.593308 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:51.593322 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.593881 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.593886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:51.594070 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:51.594968 (+ 898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:51.594996 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:51.596640 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:51.598508 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:51.598515 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:51.598532 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:51.598849 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:51.599285 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:51.599289 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:51.599290 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:51.599470 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:51.599980 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:51.599987 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:51.600343 (+ 356us) client_negotiation.cc:769] Sending connection context
0808 02:18:51.600533 (+ 190us) client_negotiation.cc:240] Negotiation successful
0808 02:18:51.600871 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":157,"threads_started":1}
I20260808 02:18:51.603502 8007 heartbeater.cc:347] Connected to a master server at 127.24.79.254:45651
I20260808 02:18:51.603824 8007 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:51.604431 8007 heartbeater.cc:511] Master 127.24.79.254:45651 requested a full tablet report, sending...
I20260808 02:18:51.605769 7519 ts_manager.cc:194] Registered new tserver with Master: 0f2234ed0df443649f0ad2b549a22782 (127.24.79.194:43473)
I20260808 02:18:51.606652 7519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.194@KRBTEST.COM'} at 127.24.79.194:52733
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
I20260808 02:18:51.619191 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:45651
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:18:51.744879 8015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:18:51.745796 8015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:18:51.745960 8015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:18:51.751111 8015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:18:51.751216 8015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:18:51.751307 8015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:18:51.757256 8015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:45651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:18:51.758842 8015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:18:51.760160 8015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:18:51.771849 8020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:51.771849 8026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:18:51.771849 8022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:18:51.772385 8015 server_base.cc:1034] running on GCE node
I20260808 02:18:51.772897 8015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:18:51.773746 8015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:18:51.774989 8015 hybrid_clock.cc:648] HybridClock initialized: now 1786155531774965 us; error 38 us; skew 500 ppm
Aug 08 02:18:51 dist-test-slave-1vmg krb5kdc[7456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155531, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:18:51.778290 8015 init.cc:377] Logged in from keytab as kudu/127.24.79.195@KRBTEST.COM (short username kudu)
I20260808 02:18:51.779419 8015 webserver.cc:460] Webserver started at http://127.24.79.195:36283/ using document root <none> and password file <none>
I20260808 02:18:51.780097 8015 fs_manager.cc:359] Metadata directory not provided
I20260808 02:18:51.780174 8015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:18:51.780404 8015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:18:51.782953 8015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "664b5ac58bbe42129df04e2685b5ff4e"
format_stamp: "Formatted at 2026-08-08 02:18:51 on dist-test-slave-1vmg"
server_key: "9b033d7d14c774825ed1bf65df97357f"
server_key_iv: "0b8e1b19e8f091b78ed4319d4d496ae8"
server_key_version: "encryptionkey@0"
I20260808 02:18:51.783651 8015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "664b5ac58bbe42129df04e2685b5ff4e"
format_stamp: "Formatted at 2026-08-08 02:18:51 on dist-test-slave-1vmg"
server_key: "9b033d7d14c774825ed1bf65df97357f"
server_key_iv: "0b8e1b19e8f091b78ed4319d4d496ae8"
server_key_version: "encryptionkey@0"
I20260808 02:18:51.788378 8015 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:18:51.791775 8037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:51.793370 8015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:18:51.793545 8015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "664b5ac58bbe42129df04e2685b5ff4e"
format_stamp: "Formatted at 2026-08-08 02:18:51 on dist-test-slave-1vmg"
server_key: "9b033d7d14c774825ed1bf65df97357f"
server_key_iv: "0b8e1b19e8f091b78ed4319d4d496ae8"
server_key_version: "encryptionkey@0"
I20260808 02:18:51.793689 8015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:18:51.813725 8015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:18:51.817762 8015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:18:51.817999 8015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:18:51.818693 8015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:18:51.819738 8015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:18:51.819818 8015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:51.819864 8015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:18:51.819878 8015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:18:51.829774 8015 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:44697
I20260808 02:18:51.829815 8207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:44697 every 8 connection(s)
I20260808 02:18:51.830874 8015 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:18:51.838511 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 8015
I20260808 02:18:51.838630 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:18:51.838966 24895 external_mini_cluster.cc:1428] Setting key b12917573eed5ea874fb954ff5bd1f55
Aug 08 02:18:51 dist-test-slave-1vmg krb5kdc[7456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155531, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:18:51.847106 7572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:51.833149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:42631 (local address 127.24.79.254:45651)
0808 02:18:51.833321 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:51.833325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:51.834394 (+ 1069us) server_negotiation.cc:407] Connection header received
0808 02:18:51.836086 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:51.836090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:51.836163 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:51.836242 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:51.837686 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.838248 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.839675 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.839974 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.840075 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:51.842966 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:51.842970 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:51.842973 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:51.843009 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:51.844537 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:51.845215 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:51.845219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:51.845221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:51.845318 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:51.845742 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:51.845746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:51.845748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:51.845973 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:51.846067 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:51.846726 (+ 659us) server_negotiation.cc:299] Negotiation successful
0808 02:18:51.846889 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 02:18:51.848160 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:51.833599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45651 (local address 127.24.79.195:42631)
0808 02:18:51.834175 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:51.834227 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:51.835706 (+ 1479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:51.836448 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:51.836456 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:51.836905 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:18:51.837489 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:51.837505 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.838403 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.838407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:51.839528 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:51.839541 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.840168 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.840173 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:51.840370 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:51.840992 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:18:51.841014 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:18:51.842747 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:18:51.844735 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:51.844744 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:51.844764 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:51.845098 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:51.845418 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:18:51.845422 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:18:51.845424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:18:51.845636 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:18:51.846096 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:18:51.846105 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:18:51.846419 (+ 314us) client_negotiation.cc:769] Sending connection context
0808 02:18:51.846633 (+ 214us) client_negotiation.cc:240] Negotiation successful
0808 02:18:51.847108 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":163,"threads_started":1}
I20260808 02:18:51.849815 8208 heartbeater.cc:347] Connected to a master server at 127.24.79.254:45651
I20260808 02:18:51.850189 8208 heartbeater.cc:464] Registering TS with master...
I20260808 02:18:51.850848 8208 heartbeater.cc:511] Master 127.24.79.254:45651 requested a full tablet report, sending...
I20260808 02:18:51.852293 7519 ts_manager.cc:194] Registered new tserver with Master: 664b5ac58bbe42129df04e2685b5ff4e (127.24.79.195:44697)
I20260808 02:18:51.853008 7519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.195@KRBTEST.COM'} at 127.24.79.195:42631
I20260808 02:18:51.853825 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:18:51.865028 7572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:51.856182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47484 (local address 127.24.79.254:45651)
0808 02:18:51.856458 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:51.856462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:51.856489 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:18:51.856611 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:51.856614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:51.856822 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:51.857015 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:18:51.857839 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.858447 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.859479 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.859725 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.859832 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:51.861359 (+ 1527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:18:51.861362 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:18:51.861364 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:18:51.861394 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:18:51.862959 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:51.863504 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:51.863508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:51.863509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:51.863617 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:18:51.863924 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:18:51.863928 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:18:51.863929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:18:51.864255 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:18:51.864389 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:51.864757 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 02:18:51.864890 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260808 02:18:51.869081 7519 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47484:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:18:51.872571 7519 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:18:51.889144 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:51.883264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:44697 (local address 127.0.0.1:49426)
0808 02:18:51.884230 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:51.884244 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:51.884327 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:51.884933 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:51.884937 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:51.884970 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:51.885098 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:51.885106 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.886362 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.886366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:51.887566 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:51.887578 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.888484 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.888488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:51.888612 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:51.888655 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 02:18:51.888785 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 02:18:51.888944 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":903,"mutex_wait_us":5,"spinlock_wait_cycles":896,"thread_start_us":107,"threads_started":1}
I20260808 02:18:51.889293 8226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:51.883060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44292 (local address 127.24.79.194:43473)
0808 02:18:51.884270 (+ 1210us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:51.884283 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:51.884310 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:18:51.884395 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:51.884401 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:51.884715 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:51.884871 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:51.885202 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.886481 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.887777 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.888653 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.888787 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:51.888956 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:51.889012 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 02:18:51.889120 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1097,"thread_start_us":143,"threads_started":1}
I20260808 02:18:51.889497 8232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:51.883864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49426 (local address 127.24.79.195:44697)
0808 02:18:51.884436 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:51.884448 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:51.884474 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:18:51.884557 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:51.884562 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:51.884755 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:51.885266 (+ 511us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:51.885412 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.886191 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.887691 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.888339 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.888451 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:51.888657 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:51.889231 (+ 574us) server_negotiation.cc:299] Negotiation successful
0808 02:18:51.889350 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":170,"threads_started":1}
I20260808 02:18:51.890549 8230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:51.883778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:39113 (local address 127.0.0.1:46548)
0808 02:18:51.884516 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:51.884527 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:51.884608 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:51.885129 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:51.885132 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:51.885149 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:51.885220 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:51.885225 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.886639 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.886642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:51.888777 (+ 2135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:51.888788 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.890132 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.890136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:51.890231 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:51.890258 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 02:18:51.890312 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 02:18:51.890389 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":16,"thread_start_us":108,"threads_started":1}
I20260808 02:18:51.890617 8233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:51.884216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46548 (local address 127.24.79.193:39113)
0808 02:18:51.884597 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:51.884603 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:51.884622 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:18:51.884688 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:51.884693 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:51.884859 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:51.884993 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:51.885345 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.886492 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.888931 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.889994 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.890111 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:51.890282 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:51.890360 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 02:18:51.890464 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":75,"threads_started":1}
I20260808 02:18:51.889144 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:51.882870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:43473 (local address 127.0.0.1:44292)
0808 02:18:51.883703 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:51.883766 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:51.883928 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:51.884877 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:51.884884 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:51.884907 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:51.885085 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:51.885092 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.886639 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.886642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:51.887623 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:51.887635 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:51.888793 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:51.888797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:51.888883 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:51.888906 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:18:51.888963 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 02:18:51.889018 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"mutex_wait_us":14,"spinlock_wait_cycles":5888,"thread_start_us":121,"threads_started":1}
I20260808 02:18:51.893469 8118 tablet_service.cc:1467] Processing CreateTablet for tablet 10dd207a09da4e4295832c5e3226690b (DEFAULT_TABLE table=test-table [id=068e53225a0a45c7af6015ff74d3e841]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:51.893935 7715 tablet_service.cc:1467] Processing CreateTablet for tablet 10dd207a09da4e4295832c5e3226690b (DEFAULT_TABLE table=test-table [id=068e53225a0a45c7af6015ff74d3e841]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:51.895153 8118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10dd207a09da4e4295832c5e3226690b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:51.895148 7715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10dd207a09da4e4295832c5e3226690b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:51.894941 7911 tablet_service.cc:1467] Processing CreateTablet for tablet 10dd207a09da4e4295832c5e3226690b (DEFAULT_TABLE table=test-table [id=068e53225a0a45c7af6015ff74d3e841]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:18:51.895936 7911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10dd207a09da4e4295832c5e3226690b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:18:51.902807 8234 tablet_bootstrap.cc:492] T 10dd207a09da4e4295832c5e3226690b P 0f2234ed0df443649f0ad2b549a22782: Bootstrap starting.
I20260808 02:18:51.903831 8236 tablet_bootstrap.cc:492] T 10dd207a09da4e4295832c5e3226690b P 9721a5cf82674929946d83d6b7dca403: Bootstrap starting.
I20260808 02:18:51.905750 8236 tablet_bootstrap.cc:654] T 10dd207a09da4e4295832c5e3226690b P 9721a5cf82674929946d83d6b7dca403: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:51.905939 8234 tablet_bootstrap.cc:654] T 10dd207a09da4e4295832c5e3226690b P 0f2234ed0df443649f0ad2b549a22782: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:51.906688 8236 log.cc:824] T 10dd207a09da4e4295832c5e3226690b P 9721a5cf82674929946d83d6b7dca403: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:51.906759 8234 log.cc:824] T 10dd207a09da4e4295832c5e3226690b P 0f2234ed0df443649f0ad2b549a22782: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:51.907414 8237 tablet_bootstrap.cc:492] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e: Bootstrap starting.
I20260808 02:18:51.909236 8236 tablet_bootstrap.cc:492] T 10dd207a09da4e4295832c5e3226690b P 9721a5cf82674929946d83d6b7dca403: No bootstrap required, opened a new log
I20260808 02:18:51.909495 8236 ts_tablet_manager.cc:1397] T 10dd207a09da4e4295832c5e3226690b P 9721a5cf82674929946d83d6b7dca403: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 02:18:51.910552 8237 tablet_bootstrap.cc:654] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e: Neither blocks nor log segments found. Creating new log.
I20260808 02:18:51.911113 8234 tablet_bootstrap.cc:492] T 10dd207a09da4e4295832c5e3226690b P 0f2234ed0df443649f0ad2b549a22782: No bootstrap required, opened a new log
I20260808 02:18:51.911345 8234 ts_tablet_manager.cc:1397] T 10dd207a09da4e4295832c5e3226690b P 0f2234ed0df443649f0ad2b549a22782: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 02:18:51.911608 8237 log.cc:824] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:18:51.914268 8236 raft_consensus.cc:348] T 10dd207a09da4e4295832c5e3226690b P 9721a5cf82674929946d83d6b7dca403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f2234ed0df443649f0ad2b549a22782" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 43473 } } peers { permanent_uuid: "9721a5cf82674929946d83d6b7dca403" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 39113 } } peers { permanent_uuid: "664b5ac58bbe42129df04e2685b5ff4e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44697 } }
I20260808 02:18:51.914597 8236 raft_consensus.cc:374] T 10dd207a09da4e4295832c5e3226690b P 9721a5cf82674929946d83d6b7dca403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:51.914665 8236 raft_consensus.cc:729] T 10dd207a09da4e4295832c5e3226690b P 9721a5cf82674929946d83d6b7dca403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9721a5cf82674929946d83d6b7dca403, State: Initialized, Role: FOLLOWER
I20260808 02:18:51.915062 8237 tablet_bootstrap.cc:492] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e: No bootstrap required, opened a new log
I20260808 02:18:51.915016 8234 raft_consensus.cc:348] T 10dd207a09da4e4295832c5e3226690b P 0f2234ed0df443649f0ad2b549a22782 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f2234ed0df443649f0ad2b549a22782" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 43473 } } peers { permanent_uuid: "9721a5cf82674929946d83d6b7dca403" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 39113 } } peers { permanent_uuid: "664b5ac58bbe42129df04e2685b5ff4e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44697 } }
I20260808 02:18:51.915302 8237 ts_tablet_manager.cc:1397] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 02:18:51.915311 8234 raft_consensus.cc:374] T 10dd207a09da4e4295832c5e3226690b P 0f2234ed0df443649f0ad2b549a22782 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:51.915243 8236 consensus_queue.cc:260] T 10dd207a09da4e4295832c5e3226690b P 9721a5cf82674929946d83d6b7dca403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f2234ed0df443649f0ad2b549a22782" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 43473 } } peers { permanent_uuid: "9721a5cf82674929946d83d6b7dca403" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 39113 } } peers { permanent_uuid: "664b5ac58bbe42129df04e2685b5ff4e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44697 } }
I20260808 02:18:51.915381 8234 raft_consensus.cc:729] T 10dd207a09da4e4295832c5e3226690b P 0f2234ed0df443649f0ad2b549a22782 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f2234ed0df443649f0ad2b549a22782, State: Initialized, Role: FOLLOWER
I20260808 02:18:51.915808 8234 consensus_queue.cc:260] T 10dd207a09da4e4295832c5e3226690b P 0f2234ed0df443649f0ad2b549a22782 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f2234ed0df443649f0ad2b549a22782" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 43473 } } peers { permanent_uuid: "9721a5cf82674929946d83d6b7dca403" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 39113 } } peers { permanent_uuid: "664b5ac58bbe42129df04e2685b5ff4e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44697 } }
I20260808 02:18:51.916437 8236 ts_tablet_manager.cc:1428] T 10dd207a09da4e4295832c5e3226690b P 9721a5cf82674929946d83d6b7dca403: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 02:18:51.916674 8234 ts_tablet_manager.cc:1428] T 10dd207a09da4e4295832c5e3226690b P 0f2234ed0df443649f0ad2b549a22782: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 02:18:51.916896 7813 heartbeater.cc:503] Master 127.24.79.254:45651 was elected leader, sending a full tablet report...
I20260808 02:18:51.917024 8007 heartbeater.cc:503] Master 127.24.79.254:45651 was elected leader, sending a full tablet report...
I20260808 02:18:51.920037 8237 raft_consensus.cc:348] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f2234ed0df443649f0ad2b549a22782" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 43473 } } peers { permanent_uuid: "9721a5cf82674929946d83d6b7dca403" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 39113 } } peers { permanent_uuid: "664b5ac58bbe42129df04e2685b5ff4e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44697 } }
I20260808 02:18:51.920351 8237 raft_consensus.cc:374] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:18:51.920413 8237 raft_consensus.cc:729] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 664b5ac58bbe42129df04e2685b5ff4e, State: Initialized, Role: FOLLOWER
I20260808 02:18:51.920909 8237 consensus_queue.cc:260] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f2234ed0df443649f0ad2b549a22782" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 43473 } } peers { permanent_uuid: "9721a5cf82674929946d83d6b7dca403" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 39113 } } peers { permanent_uuid: "664b5ac58bbe42129df04e2685b5ff4e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44697 } }
I20260808 02:18:51.921757 8208 heartbeater.cc:503] Master 127.24.79.254:45651 was elected leader, sending a full tablet report...
I20260808 02:18:51.922636 8237 ts_tablet_manager.cc:1428] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 02:18:52.024739 8246 raft_consensus.cc:482] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:18:52.025022 8246 raft_consensus.cc:504] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f2234ed0df443649f0ad2b549a22782" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 43473 } } peers { permanent_uuid: "9721a5cf82674929946d83d6b7dca403" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 39113 } } peers { permanent_uuid: "664b5ac58bbe42129df04e2685b5ff4e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44697 } }
I20260808 02:18:52.026599 8246 leader_election.cc:288] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f2234ed0df443649f0ad2b549a22782 (127.24.79.194:43473), 9721a5cf82674929946d83d6b7dca403 (127.24.79.193:39113)
I20260808 02:18:52.031456 8226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:52.027095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:44977 (local address 127.24.79.194:43473)
0808 02:18:52.027269 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:52.027274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:52.027298 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:18:52.027492 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:52.027496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:52.027558 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:52.027647 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:52.028076 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:52.028728 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:52.030125 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:52.030781 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:52.030948 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:52.031016 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:52.031227 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 02:18:52.031314 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 02:18:52.031458 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:52.027027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:43473 (local address 127.24.79.195:44977)
0808 02:18:52.027192 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:52.027209 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:52.027320 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:52.027793 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:52.027797 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:52.027819 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:52.027946 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:52.027953 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:52.028852 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:52.028856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:52.029956 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:52.029968 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:52.030947 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:52.030951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:52.031079 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:52.031109 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 02:18:52.031192 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 02:18:52.031296 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 02:18:52.032192 8233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:52.027551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:37635 (local address 127.24.79.193:39113)
0808 02:18:52.027683 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:52.027687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:52.027952 (+ 265us) server_negotiation.cc:407] Connection header received
0808 02:18:52.028049 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:52.028053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:52.028108 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:52.028211 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:18:52.028522 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:52.029247 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:52.030717 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:52.031362 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:52.031482 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:52.031536 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:52.031956 (+ 420us) server_negotiation.cc:299] Negotiation successful
0808 02:18:52.032057 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 02:18:52.032501 7946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10dd207a09da4e4295832c5e3226690b" candidate_uuid: "664b5ac58bbe42129df04e2685b5ff4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f2234ed0df443649f0ad2b549a22782" is_pre_election: true
I20260808 02:18:52.032624 8248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:52.027342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:39113 (local address 127.24.79.195:37635)
0808 02:18:52.027841 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:52.027855 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:18:52.027954 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:18:52.028245 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:18:52.028248 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:18:52.028272 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:18:52.028391 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:52.028397 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:52.029403 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:52.029407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:52.030536 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:18:52.030549 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:18:52.031632 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:18:52.031636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:18:52.031790 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:52.031818 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 02:18:52.032372 (+ 554us) client_negotiation.cc:240] Negotiation successful
0808 02:18:52.032466 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":161,"threads_started":1}
I20260808 02:18:52.032917 7946 raft_consensus.cc:2454] T 10dd207a09da4e4295832c5e3226690b P 0f2234ed0df443649f0ad2b549a22782 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 664b5ac58bbe42129df04e2685b5ff4e in term 0.
I20260808 02:18:52.033394 7738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10dd207a09da4e4295832c5e3226690b" candidate_uuid: "664b5ac58bbe42129df04e2685b5ff4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9721a5cf82674929946d83d6b7dca403" is_pre_election: true
I20260808 02:18:52.033672 8043 leader_election.cc:302] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f2234ed0df443649f0ad2b549a22782, 664b5ac58bbe42129df04e2685b5ff4e; no voters:
I20260808 02:18:52.033737 7738 raft_consensus.cc:2454] T 10dd207a09da4e4295832c5e3226690b P 9721a5cf82674929946d83d6b7dca403 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 664b5ac58bbe42129df04e2685b5ff4e in term 0.
I20260808 02:18:52.033981 8246 raft_consensus.cc:2781] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:18:52.034097 8246 raft_consensus.cc:482] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:18:52.034256 8246 raft_consensus.cc:3037] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:52.035583 8246 raft_consensus.cc:504] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f2234ed0df443649f0ad2b549a22782" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 43473 } } peers { permanent_uuid: "9721a5cf82674929946d83d6b7dca403" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 39113 } } peers { permanent_uuid: "664b5ac58bbe42129df04e2685b5ff4e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44697 } }
I20260808 02:18:52.036255 8246 leader_election.cc:288] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e [CANDIDATE]: Term 1 election: Requested vote from peers 0f2234ed0df443649f0ad2b549a22782 (127.24.79.194:43473), 9721a5cf82674929946d83d6b7dca403 (127.24.79.193:39113)
I20260808 02:18:52.036808 7946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10dd207a09da4e4295832c5e3226690b" candidate_uuid: "664b5ac58bbe42129df04e2685b5ff4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f2234ed0df443649f0ad2b549a22782"
I20260808 02:18:52.036888 7738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10dd207a09da4e4295832c5e3226690b" candidate_uuid: "664b5ac58bbe42129df04e2685b5ff4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9721a5cf82674929946d83d6b7dca403"
I20260808 02:18:52.036958 7946 raft_consensus.cc:3037] T 10dd207a09da4e4295832c5e3226690b P 0f2234ed0df443649f0ad2b549a22782 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:52.037081 7738 raft_consensus.cc:3037] T 10dd207a09da4e4295832c5e3226690b P 9721a5cf82674929946d83d6b7dca403 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:18:52.038475 7946 raft_consensus.cc:2454] T 10dd207a09da4e4295832c5e3226690b P 0f2234ed0df443649f0ad2b549a22782 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 664b5ac58bbe42129df04e2685b5ff4e in term 1.
I20260808 02:18:52.038859 8043 leader_election.cc:302] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f2234ed0df443649f0ad2b549a22782, 664b5ac58bbe42129df04e2685b5ff4e; no voters:
I20260808 02:18:52.039108 8246 raft_consensus.cc:2781] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:18:52.039036 7738 raft_consensus.cc:2454] T 10dd207a09da4e4295832c5e3226690b P 9721a5cf82674929946d83d6b7dca403 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 664b5ac58bbe42129df04e2685b5ff4e in term 1.
I20260808 02:18:52.039424 8246 raft_consensus.cc:686] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e [term 1 LEADER]: Becoming Leader. State: Replica: 664b5ac58bbe42129df04e2685b5ff4e, State: Running, Role: LEADER
I20260808 02:18:52.040048 8246 consensus_queue.cc:237] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f2234ed0df443649f0ad2b549a22782" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 43473 } } peers { permanent_uuid: "9721a5cf82674929946d83d6b7dca403" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 39113 } } peers { permanent_uuid: "664b5ac58bbe42129df04e2685b5ff4e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44697 } }
I20260808 02:18:52.043897 7518 catalog_manager.cc:5495] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e reported cstate change: term changed from 0 to 1, leader changed from <none> to 664b5ac58bbe42129df04e2685b5ff4e (127.24.79.195). New cstate: current_term: 1 leader_uuid: "664b5ac58bbe42129df04e2685b5ff4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f2234ed0df443649f0ad2b549a22782" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 43473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9721a5cf82674929946d83d6b7dca403" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 39113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "664b5ac58bbe42129df04e2685b5ff4e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44697 } health_report { overall_health: HEALTHY } } }
W20260808 02:18:52.082103 8210 tablet.cc:2369] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:18:52.086274 8008 tablet.cc:2369] T 10dd207a09da4e4295832c5e3226690b P 0f2234ed0df443649f0ad2b549a22782: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:18:52.086824 8232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:52.082465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49428 (local address 127.24.79.195:44697)
0808 02:18:52.082622 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 02:18:52.082628 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:18:52.082648 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:18:52.082804 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:18:52.082807 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:18:52.082879 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:18:52.082998 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:18:52.083328 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:52.083904 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:52.085043 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:18:52.085347 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:18:52.085451 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:18:52.085783 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:18:52.086353 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:18:52.086493 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 02:18:52.086601 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 02:18:52.086673 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:18:52.094416 7738 raft_consensus.cc:1261] T 10dd207a09da4e4295832c5e3226690b P 9721a5cf82674929946d83d6b7dca403 [term 1 FOLLOWER]: Refusing update from remote peer 664b5ac58bbe42129df04e2685b5ff4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:18:52.094416 7946 raft_consensus.cc:1261] T 10dd207a09da4e4295832c5e3226690b P 0f2234ed0df443649f0ad2b549a22782 [term 1 FOLLOWER]: Refusing update from remote peer 664b5ac58bbe42129df04e2685b5ff4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:18:52.095639 8250 consensus_queue.cc:1035] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f2234ed0df443649f0ad2b549a22782" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:18:52.095896 8246 consensus_queue.cc:1035] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9721a5cf82674929946d83d6b7dca403" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 39113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 02:18:52.100185 7815 tablet.cc:2369] T 10dd207a09da4e4295832c5e3226690b P 9721a5cf82674929946d83d6b7dca403: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:18:52.105721 8255 mvcc.cc:205] Tried to move back new op lower bound from 7316093059447910400 to 7316093059239256064. Current Snapshot: MvccSnapshot[applied={T|T < 7316093059447910400}]
I20260808 02:18:52.106721 8258 mvcc.cc:205] Tried to move back new op lower bound from 7316093059447910400 to 7316093059239256064. Current Snapshot: MvccSnapshot[applied={T|T < 7316093059447910400}]
I20260808 02:18:52.106962 8256 mvcc.cc:205] Tried to move back new op lower bound from 7316093059447910400 to 7316093059239256064. Current Snapshot: MvccSnapshot[applied={T|T < 7316093059447910400}]
I20260808 02:18:52.116676 7519 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:18:52.116905 7519 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:18:52.119930 7519 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 80bcb614f6af4d34acfa6874fee5dd03: Sending DeleteTablet for 3 replicas of tablet 10dd207a09da4e4295832c5e3226690b
I20260808 02:18:52.121037 7715 tablet_service.cc:1514] Processing DeleteTablet for tablet 10dd207a09da4e4295832c5e3226690b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:52 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:46548
I20260808 02:18:52.121034 8118 tablet_service.cc:1514] Processing DeleteTablet for tablet 10dd207a09da4e4295832c5e3226690b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:52 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:49426
I20260808 02:18:52.121039 7911 tablet_service.cc:1514] Processing DeleteTablet for tablet 10dd207a09da4e4295832c5e3226690b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:18:52 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:44292
I20260808 02:18:52.121929 8274 tablet_replica.cc:330] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e: stopping tablet replica
I20260808 02:18:52.122372 8274 raft_consensus.cc:2229] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:18:52.122488 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 7614
I20260808 02:18:52.123014 8274 raft_consensus.cc:2258] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:18:52.128445 8274 ts_tablet_manager.cc:1905] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:18:52.131222 8276 tablet_replica.cc:330] T 10dd207a09da4e4295832c5e3226690b P 0f2234ed0df443649f0ad2b549a22782: stopping tablet replica
I20260808 02:18:52.131711 8276 raft_consensus.cc:2229] T 10dd207a09da4e4295832c5e3226690b P 0f2234ed0df443649f0ad2b549a22782 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:18:52.132117 8276 raft_consensus.cc:2258] T 10dd207a09da4e4295832c5e3226690b P 0f2234ed0df443649f0ad2b549a22782 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:18:52.134044 8274 ts_tablet_manager.cc:1918] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:18:52.134181 8274 log.cc:1196] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/10dd207a09da4e4295832c5e3226690b
I20260808 02:18:52.134562 8274 ts_tablet_manager.cc:1939] T 10dd207a09da4e4295832c5e3226690b P 664b5ac58bbe42129df04e2685b5ff4e: Deleting consensus metadata
I20260808 02:18:52.136747 7504 catalog_manager.cc:4841] TS 664b5ac58bbe42129df04e2685b5ff4e (127.24.79.195:44697): tablet 10dd207a09da4e4295832c5e3226690b (table test-table [id=068e53225a0a45c7af6015ff74d3e841]) successfully deleted
I20260808 02:18:52.139902 8276 ts_tablet_manager.cc:1905] T 10dd207a09da4e4295832c5e3226690b P 0f2234ed0df443649f0ad2b549a22782: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:18:52.142553 8276 ts_tablet_manager.cc:1918] T 10dd207a09da4e4295832c5e3226690b P 0f2234ed0df443649f0ad2b549a22782: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:18:52.142676 8276 log.cc:1196] T 10dd207a09da4e4295832c5e3226690b P 0f2234ed0df443649f0ad2b549a22782: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/10dd207a09da4e4295832c5e3226690b
I20260808 02:18:52.143018 8276 ts_tablet_manager.cc:1939] T 10dd207a09da4e4295832c5e3226690b P 0f2234ed0df443649f0ad2b549a22782: Deleting consensus metadata
I20260808 02:18:52.145143 7503 catalog_manager.cc:4841] TS 0f2234ed0df443649f0ad2b549a22782 (127.24.79.194:43473): tablet 10dd207a09da4e4295832c5e3226690b (table test-table [id=068e53225a0a45c7af6015ff74d3e841]) successfully deleted
W20260808 02:18:52.148904 7503 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.79.193:39113 (error 108)
I20260808 02:18:52.149008 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 7819
I20260808 02:18:52.157624 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:18:52.157193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:39113 (local address 127.0.0.1:46564)
0808 02:18:52.157383 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 02:18:52.157492 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.193:39113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260808 02:18:52.158030 7503 catalog_manager.cc:4584] TS 9721a5cf82674929946d83d6b7dca403 (127.24.79.193:39113): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 10dd207a09da4e4295832c5e3226690b: Network error: Client connection negotiation failed: client connection to 127.24.79.193:39113: connect: Connection refused (error 111)
I20260808 02:18:52.163117 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 8015
I20260808 02:18:52.194427 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 7474
2026-08-08T02:18:52Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2712 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:18:52 dist-test-slave-1vmg krb5kdc[8281](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:18:52 dist-test-slave-1vmg krb5kdc[8281](info): set up 2 sockets
Aug 08 02:18:52 dist-test-slave-1vmg krb5kdc[8281](info): commencing operation
krb5kdc: starting...
W20260808 02:18:53.358700 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:18:53 dist-test-slave-1vmg krb5kdc[8281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:18:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:18:53Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.79.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.79.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.79.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/rangeradmin_127.24.79.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.79.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/rangeradmin_127.24.79.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.79.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.79.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.79.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/rangerlookup_127.24.79.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.79.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/rangerlookup_127.24.79.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.79.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.79.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.79.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/HTTP_127.24.79.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.79.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/HTTP_127.24.79.212@KRBTEST.COM.keytab.
I20260808 02:18:53.584769 24895 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task9BOO5x/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/postgres -l logfile start
2026-08-08 02:18:56.679 UTC [8321] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 02:18:56.679 UTC [8321] LOG: listening on IPv4 address "127.24.79.212", port 57061
2026-08-08 02:18:56.685 UTC [8321] LOG: listening on Unix socket "/tmp/.s.PGSQL.57061"
2026-08-08 02:18:56.710 UTC [8324] LOG: database system was shut down at 2026-08-08 02:18:54 UTC
2026-08-08 02:18:56.715 UTC [8321] LOG: database system is ready to accept connections
I20260808 02:18:57.796491 24895 mini_postgres.cc:96] Postgres bound to 57061
2026-08-08 02:18:57.807 UTC [8336] FATAL: database "slave" does not exist
127.24.79.212:57061 - accepting connections
I20260808 02:18:57.808678 24895 mini_ranger.cc:161] Starting Ranger...
I20260808 02:18:57.831121 24895 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 02:18:58.542618 24895 mini_ranger.cc:87] Created ranger Postgres database
I20260808 02:18:58.542732 24895 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ranger-admin
2026-08-08 02:18:58,911 [I] DB FLAVOR :POSTGRES
2026-08-08 02:18:58,911 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 02:18:58,911 [I] Checking connection..
2026-08-08 02:18:58,912 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:57061/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 02:18:59,217 [I] Checking connection passed.
2026-08-08 02:18:59,217 [I] --------- Verifying version history table ---------
2026-08-08 02:18:59,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:57061/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:18:59,513 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 02:18:59,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:57061/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:18:59,796 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 02:18:59,796 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 02:18:59,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:57061/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 02:19:00.077 UTC [8441] WARNING: there is no transaction in progress
2026-08-08 02:19:00,088 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 02:19:00,089 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:57061/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:19:00,380 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 02:19:00,381 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 02:19:00,381 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:57061/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 02:19:00,660 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:57061/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 02:19:00,936 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:57061/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1vmg.c.gcp-upstream.internal','N') ;"
2026-08-08 02:19:01,215 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:57061/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 02:19:01,494 [I] Table x_portal_user does not exist in database ranger
2026-08-08 02:19:01,494 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:57061/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 02:19:01,785 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 02:19:01,785 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:57061/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 02:19:02,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:57061/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 02:19:02,345 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 02:19:02,345 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:57061/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 02:19:03.663 UTC [8602] WARNING: there is no transaction in progress
2026-08-08 02:19:03.687 UTC [8602] WARNING: there is no transaction in progress
2026-08-08 02:19:03.713 UTC [8602] WARNING: there is no transaction in progress
2026-08-08 02:19:03.739 UTC [8602] WARNING: there is no transaction in progress
2026-08-08 02:19:03.769 UTC [8602] WARNING: there is no transaction in progress
2026-08-08 02:19:03.797 UTC [8602] WARNING: there is no transaction in progress
2026-08-08 02:19:03.875 UTC [8602] WARNING: there is no transaction in progress
2026-08-08 02:19:03.885 UTC [8602] WARNING: there is no transaction in progress
2026-08-08 02:19:03.907 UTC [8602] WARNING: there is no transaction in progress
2026-08-08 02:19:03.922 UTC [8602] WARNING: there is no transaction in progress
2026-08-08 02:19:03.936 UTC [8602] WARNING: there is no transaction in progress
2026-08-08 02:19:03.956 UTC [8602] WARNING: there is no transaction in progress
2026-08-08 02:19:03.969 UTC [8602] WARNING: there is no transaction in progress
2026-08-08 02:19:03.978 UTC [8602] WARNING: there is no transaction in progress
2026-08-08 02:19:03.988 UTC [8602] WARNING: there is no transaction in progress
2026-08-08 02:19:03.999 UTC [8602] WARNING: there is no transaction in progress
2026-08-08 02:19:05,067 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 02:19:05,067 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:57061/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 02:19:05,347 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 02:19:05,347 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:57061/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 02:19:05,647 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 02:19:05,647 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:57061/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 02:19:05,948 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 02:19:05,948 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:57061/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 02:19:06,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:57061/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 02:19:06,505 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:57061/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1vmg.c.gcp-upstream.internal';"
2026-08-08 02:19:06,783 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 02:19:06,784 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:57061/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 02:19:07,059 [I] DB_PATCHES have already been applied
I20260808 02:19:07.066275 24895 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ranger-admin:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task9BOO5x/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 02:19:07.066358 24895 mini_ranger.cc:193] Using host: 127.24.79.212
I20260808 02:19:07.071834 24895 mini_ranger.cc:227] Ranger admin URL: http://127.24.79.212:57473
Aug 08, 2026 2:19:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:19:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:19:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 2:19:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 2:19:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 2:19:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 2:19:08 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 02:19:08.327390 24895 mini_ranger.cc:160] Time spent starting Ranger: real 10.519s user 0.008s sys 0.007s
Aug 08, 2026 2:19:31 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 2:19:31 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 2:19:31 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 2:19:31 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 2:19:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 2:19:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 2:19:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 2:19:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 2:19:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 2:19:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 2:19:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 2:19:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 2:19:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 2:19:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 2:19:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 2:19:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 2:19:31 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 2:19:33 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 02:19:33.735994 24895 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.79.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.79.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.79.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/rangerkms_127.24.79.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.79.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/rangerkms_127.24.79.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.79.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.79.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.79.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/HTTP_127.24.79.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.79.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/HTTP_127.24.79.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 02:19:33 dist-test-slave-1vmg krb5kdc[8281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155573, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 02:19:33.871346 24895 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 02:19:33.886752 24895 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 02:19:35.322572 24895 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 02:19:35.322737 24895 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ranger-kms
I20260808 02:19:35.322868 24895 mini_ranger_kms.cc:227] Using postgres at 127.24.79.212:57061
2026-08-08 02:19:35,849 [I] DB FLAVOR :POSTGRES
2026-08-08 02:19:35,850 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 02:19:35,850 [I] Checking connection
2026-08-08 02:19:35,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:57061/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 02:19:36,172 [I] connection success
2026-08-08 02:19:36,173 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 02:19:36,173 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 02:19:36,173 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:57061/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 02:19:36,467 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 02:19:36,468 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 02:19:36,468 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 02:19:36,468 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:57061/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 02:19:36,804 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 02:19:37.514667 24895 mini_ranger_kms.cc:310] Created kms service
I20260808 02:19:37.804344 24895 mini_ranger_kms.cc:326] Created kudu user
I20260808 02:19:37.863349 24895 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 02:19:37 dist-test-slave-1vmg krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155573, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.79.212@KRBTEST.COM
I20260808 02:19:38.414887 24895 mini_ranger_kms.cc:384] Added ranger policy
I20260808 02:19:38.415294 24895 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ranger-kms:/tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 02:19:38.415331 24895 mini_ranger_kms.cc:240] Using host: 127.24.79.212
I20260808 02:19:38.420745 24895 mini_ranger_kms.cc:276] Ranger KMS PID: 8880
I20260808 02:19:38.420946 24895 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.79.212:47441
Aug 08, 2026 2:19:39 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:19:39 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:19:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 2:19:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 2:19:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 2:19:40 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 02:19:40 dist-test-slave-1vmg krb5kdc[8281](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786155580, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.79.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:19:41.083863 24895 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 7.213s user 0.015s sys 0.257s
I20260808 02:19:41.084034 24895 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 02:19:41.084182 24895 mini_ranger_kms.cc:401] 127.24.79.212:47441/kms/v1/keys
Aug 08 02:19:41 dist-test-slave-1vmg krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155580, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.79.212@KRBTEST.COM for HTTP/127.24.79.212@KRBTEST.COM
Aug 08 02:19:41 dist-test-slave-1vmg krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155580, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.79.212@KRBTEST.COM for HTTP/127.24.79.212@KRBTEST.COM
Aug 08 02:19:44 dist-test-slave-1vmg krb5kdc[8281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:19:44.636204 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:41773
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:36265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:41773
--ranger_config_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.79.212:47441/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:19:44.772068 8955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:19:44.772580 8955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:19:44.772640 8955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:19:44.777475 8955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:19:44.777595 8955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:19:44.777616 8955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:19:44.777631 8955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:19:44.777647 8955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:19:44.784140 8955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:36265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.79.212:47441/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:41773
--ranger_config_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:41773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:19:44.785732 8955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:19:44.787230 8955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:19:44.798766 8966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:19:44.798852 8955 server_base.cc:1034] running on GCE node
W20260808 02:19:44.798763 8962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:19:44.798748 8970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:19:44.799932 8955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:19:44.801352 8955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:19:44.802591 8955 hybrid_clock.cc:648] HybridClock initialized: now 1786155584802522 us; error 106 us; skew 500 ppm
Aug 08 02:19:44 dist-test-slave-1vmg krb5kdc[8281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155584, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:19:44.809923 8955 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:19:44.811319 8955 webserver.cc:460] Webserver started at http://127.24.79.254:37615/ using document root <none> and password file <none>
I20260808 02:19:44.811972 8955 fs_manager.cc:359] Metadata directory not provided
I20260808 02:19:44.812040 8955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:19:44.812283 8955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:19:44 dist-test-slave-1vmg krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155584, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for HTTP/127.24.79.212@KRBTEST.COM
I20260808 02:19:44.881524 8955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "50b9f424434744f9b531ca6b47aa5ca4"
format_stamp: "Formatted at 2026-08-08 02:19:44 on dist-test-slave-1vmg"
server_key: "69b1822e962e8ef379b6eb211b876e96"
server_key_iv: "0452a5cfbb9ed7ec61b49034034ef4d4"
server_key_version: "kuduclusterkey@0"
I20260808 02:19:44.882511 8955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "50b9f424434744f9b531ca6b47aa5ca4"
format_stamp: "Formatted at 2026-08-08 02:19:44 on dist-test-slave-1vmg"
server_key: "69b1822e962e8ef379b6eb211b876e96"
server_key_iv: "0452a5cfbb9ed7ec61b49034034ef4d4"
server_key_version: "kuduclusterkey@0"
I20260808 02:19:44.889168 8955 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260808 02:19:44.912281 8979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:19:44.914081 8955 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:19:44.914292 8955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "50b9f424434744f9b531ca6b47aa5ca4"
format_stamp: "Formatted at 2026-08-08 02:19:44 on dist-test-slave-1vmg"
server_key: "69b1822e962e8ef379b6eb211b876e96"
server_key_iv: "0452a5cfbb9ed7ec61b49034034ef4d4"
server_key_version: "kuduclusterkey@0"
I20260808 02:19:44.914436 8955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:19:44.937095 8955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:19:44.941069 8955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:19:44.941318 8955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:19:44.950168 8955 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:41773
I20260808 02:19:44.950201 9055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:41773 every 8 connection(s)
I20260808 02:19:44.951396 8955 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:19:44.954860 9056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:19:44.958330 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 8955
I20260808 02:19:44.958480 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:19:44.958815 24895 external_mini_cluster.cc:1428] Setting key 439ba804bc04a4d9539cc10b31ad44bc
I20260808 02:19:44.962963 9056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50b9f424434744f9b531ca6b47aa5ca4: Bootstrap starting.
I20260808 02:19:44.966476 9056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50b9f424434744f9b531ca6b47aa5ca4: Neither blocks nor log segments found. Creating new log.
I20260808 02:19:44.967959 9056 log.cc:824] T 00000000000000000000000000000000 P 50b9f424434744f9b531ca6b47aa5ca4: Log is configured to *not* fsync() on all Append() calls
I20260808 02:19:44.971273 9056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50b9f424434744f9b531ca6b47aa5ca4: No bootstrap required, opened a new log
Aug 08 02:19:44 dist-test-slave-1vmg krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:19:44.976620 9056 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50b9f424434744f9b531ca6b47aa5ca4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b9f424434744f9b531ca6b47aa5ca4" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41773 } }
I20260808 02:19:44.977008 9056 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50b9f424434744f9b531ca6b47aa5ca4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:19:44.977102 9056 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50b9f424434744f9b531ca6b47aa5ca4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50b9f424434744f9b531ca6b47aa5ca4, State: Initialized, Role: FOLLOWER
I20260808 02:19:44.977806 9056 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50b9f424434744f9b531ca6b47aa5ca4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b9f424434744f9b531ca6b47aa5ca4" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41773 } }
I20260808 02:19:44.977984 9056 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50b9f424434744f9b531ca6b47aa5ca4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:19:44.978082 9056 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50b9f424434744f9b531ca6b47aa5ca4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:19:44.978219 9056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50b9f424434744f9b531ca6b47aa5ca4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:19:44.979583 9056 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50b9f424434744f9b531ca6b47aa5ca4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b9f424434744f9b531ca6b47aa5ca4" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41773 } }
I20260808 02:19:44.980032 9056 leader_election.cc:302] T 00000000000000000000000000000000 P 50b9f424434744f9b531ca6b47aa5ca4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50b9f424434744f9b531ca6b47aa5ca4; no voters:
I20260808 02:19:44.980463 9056 leader_election.cc:288] T 00000000000000000000000000000000 P 50b9f424434744f9b531ca6b47aa5ca4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:19:44.980567 9061 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50b9f424434744f9b531ca6b47aa5ca4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:19:44.980872 9061 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50b9f424434744f9b531ca6b47aa5ca4 [term 1 LEADER]: Becoming Leader. State: Replica: 50b9f424434744f9b531ca6b47aa5ca4, State: Running, Role: LEADER
I20260808 02:19:44.980962 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:19:44.960726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35448 (local address 127.24.79.254:41773)
0808 02:19:44.961408 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0808 02:19:44.961434 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 02:19:44.961491 (+ 57us) server_negotiation.cc:407] Connection header received
0808 02:19:44.962529 (+ 1038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:19:44.962564 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:19:44.963227 (+ 663us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:19:44.963726 (+ 499us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:19:44.964890 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:19:44.966805 (+ 1915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:19:44.968526 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:19:44.969380 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:19:44.969592 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:19:44.973629 (+ 4037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:19:44.973638 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:19:44.973710 (+ 72us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:19:44.973759 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:19:44.976106 (+ 2347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:19:44.976822 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:19:44.976830 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:19:44.976837 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:19:44.977009 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:19:44.977534 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:19:44.977537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:19:44.977539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:19:44.978204 (+ 665us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:19:44.978428 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 02:19:44.978945 (+ 517us) server_negotiation.cc:299] Negotiation successful
0808 02:19:44.979543 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":206,"threads_started":1}
I20260808 02:19:44.981292 9061 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50b9f424434744f9b531ca6b47aa5ca4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b9f424434744f9b531ca6b47aa5ca4" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41773 } }
I20260808 02:19:44.982167 9056 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50b9f424434744f9b531ca6b47aa5ca4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:19:44.982892 9061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50b9f424434744f9b531ca6b47aa5ca4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50b9f424434744f9b531ca6b47aa5ca4. Latest consensus state: current_term: 1 leader_uuid: "50b9f424434744f9b531ca6b47aa5ca4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b9f424434744f9b531ca6b47aa5ca4" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41773 } } }
I20260808 02:19:44.983199 9061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50b9f424434744f9b531ca6b47aa5ca4 [sys.catalog]: This master's current role is: LEADER
I20260808 02:19:44.983492 9061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50b9f424434744f9b531ca6b47aa5ca4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50b9f424434744f9b531ca6b47aa5ca4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b9f424434744f9b531ca6b47aa5ca4" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41773 } } }
I20260808 02:19:44.983670 9061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50b9f424434744f9b531ca6b47aa5ca4 [sys.catalog]: This master's current role is: LEADER
I20260808 02:19:44.984218 9072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:19:44.988224 9072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:19:44.988904 9056 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 02:19:45.002311 9072 catalog_manager.cc:1347] Generated new cluster ID: 78ad4909c5f0482a828fec1c2aeb5cac
I20260808 02:19:45.002472 9072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:19:45.020437 9072 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:19:45.022127 9072 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:19:45.031081 9072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50b9f424434744f9b531ca6b47aa5ca4: Generated new TSK 0
I20260808 02:19:45.032464 9072 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.79.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
I20260808 02:19:45.916395 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.79.212:47441/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:41773
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:36265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:19:46.060329 9119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:19:46.060698 9119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:19:46.060763 9119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:19:46.065225 9119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:19:46.065325 9119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:19:46.065409 9119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:19:46.072096 9119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:36265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.79.212:47441/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:41773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:19:46.073532 9119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:19:46.075099 9119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:19:46.089752 9128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:19:46.089833 9119 server_base.cc:1034] running on GCE node
W20260808 02:19:46.090058 9129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:19:46.089941 9132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:19:46.090752 9119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:19:46.091780 9119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:19:46.093024 9119 hybrid_clock.cc:648] HybridClock initialized: now 1786155586092963 us; error 82 us; skew 500 ppm
Aug 08 02:19:46 dist-test-slave-1vmg krb5kdc[8281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155586, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:19:46.097441 9119 init.cc:377] Logged in from keytab as kudu/127.24.79.193@KRBTEST.COM (short username kudu)
I20260808 02:19:46.098896 9119 webserver.cc:460] Webserver started at http://127.24.79.193:44249/ using document root <none> and password file <none>
I20260808 02:19:46.099843 9119 fs_manager.cc:359] Metadata directory not provided
I20260808 02:19:46.099965 9119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:19:46.100349 9119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:19:46 dist-test-slave-1vmg krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155586, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for HTTP/127.24.79.212@KRBTEST.COM
I20260808 02:19:46.123610 9119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "a71fe1c0e2a14fedb195d3b91244fcdc"
format_stamp: "Formatted at 2026-08-08 02:19:46 on dist-test-slave-1vmg"
server_key: "2f1887cb48b4b7408704880bbe04f9ad"
server_key_iv: "4505d0dee8b6cc950a0e67cdb5bb34bc"
server_key_version: "kuduclusterkey@0"
I20260808 02:19:46.124666 9119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "a71fe1c0e2a14fedb195d3b91244fcdc"
format_stamp: "Formatted at 2026-08-08 02:19:46 on dist-test-slave-1vmg"
server_key: "2f1887cb48b4b7408704880bbe04f9ad"
server_key_iv: "4505d0dee8b6cc950a0e67cdb5bb34bc"
server_key_version: "kuduclusterkey@0"
I20260808 02:19:46.131752 9119 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260808 02:19:46.153882 9143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:19:46.156242 9119 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260808 02:19:46.156584 9119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "a71fe1c0e2a14fedb195d3b91244fcdc"
format_stamp: "Formatted at 2026-08-08 02:19:46 on dist-test-slave-1vmg"
server_key: "2f1887cb48b4b7408704880bbe04f9ad"
server_key_iv: "4505d0dee8b6cc950a0e67cdb5bb34bc"
server_key_version: "kuduclusterkey@0"
I20260808 02:19:46.156877 9119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:19:46.202384 9119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:19:46.207396 9119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:19:46.207794 9119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:19:46.208921 9119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:19:46.210331 9119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:19:46.210467 9119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260808 02:19:46.210628 9119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:19:46.210721 9119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:19:46.231336 9119 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:41853
I20260808 02:19:46.233068 9119 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:19:46.242950 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 9119
I20260808 02:19:46.243096 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:19:46.243273 9309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:41853 every 8 connection(s)
I20260808 02:19:46.243485 24895 external_mini_cluster.cc:1428] Setting key 0532ade1629e9d6aad2ea221942ed387
Aug 08 02:19:46 dist-test-slave-1vmg krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155586, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:19:46.266779 9314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:19:46.239037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:35951 (local address 127.24.79.254:41773)
0808 02:19:46.251219 (+ 12182us) server_negotiation.cc:206] Beginning negotiation
0808 02:19:46.251224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:19:46.251258 (+ 34us) server_negotiation.cc:407] Connection header received
0808 02:19:46.251323 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:19:46.251327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:19:46.251401 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:19:46.251560 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:19:46.253838 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:19:46.254762 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:19:46.256858 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:19:46.257228 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:19:46.257368 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:19:46.261274 (+ 3906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:19:46.261279 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:19:46.261282 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:19:46.261322 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:19:46.263457 (+ 2135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:19:46.264457 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:19:46.264461 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:19:46.264463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:19:46.264581 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:19:46.265074 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:19:46.265079 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:19:46.265081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:19:46.265361 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:19:46.265470 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 02:19:46.266384 (+ 914us) server_negotiation.cc:299] Negotiation successful
0808 02:19:46.266561 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12052,"thread_start_us":142,"threads_started":1}
I20260808 02:19:46.269238 9312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:19:46.237511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:41773 (local address 127.24.79.193:35951)
0808 02:19:46.238318 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0808 02:19:46.238394 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0808 02:19:46.239644 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:19:46.252114 (+ 12470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:19:46.252130 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:19:46.252661 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:19:46.253575 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:19:46.253602 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:19:46.255011 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:19:46.255015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:19:46.256646 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:19:46.256665 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:19:46.257645 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:19:46.257651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:19:46.257911 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:19:46.258821 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:19:46.258848 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:19:46.260955 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:19:46.263681 (+ 2726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:19:46.263690 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:19:46.263718 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:19:46.264312 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:19:46.264709 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:19:46.264712 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:19:46.264714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:19:46.264951 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:19:46.265564 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:19:46.265574 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:19:46.266027 (+ 453us) client_negotiation.cc:769] Sending connection context
0808 02:19:46.267374 (+ 1347us) client_negotiation.cc:240] Negotiation successful
0808 02:19:46.267859 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":228,"threads_started":1}
WARNING: no policy specified for kudu/127.24.79.194@KRBTEST.COM; defaulting to no policy
I20260808 02:19:46.277758 9310 heartbeater.cc:347] Connected to a master server at 127.24.79.254:41773
I20260808 02:19:46.278117 9310 heartbeater.cc:464] Registering TS with master...
I20260808 02:19:46.279124 9310 heartbeater.cc:511] Master 127.24.79.254:41773 requested a full tablet report, sending...
I20260808 02:19:46.281767 8995 ts_manager.cc:194] Registered new tserver with Master: a71fe1c0e2a14fedb195d3b91244fcdc (127.24.79.193:41853)
I20260808 02:19:46.283644 8995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.193@KRBTEST.COM'} at 127.24.79.193:35951
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.194@KRBTEST.COM" created.
Aug 08 02:19:46 dist-test-slave-1vmg krb5kdc[8281](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786155586, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
I20260808 02:19:46.322971 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.79.212:47441/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:41773
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:36265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:19:46.541404 9320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:19:46.542452 9320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:19:46.542629 9320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:19:46.549522 9320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:19:46.549816 9320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:19:46.549989 9320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:19:46.559015 9320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:36265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.79.212:47441/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:41773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:19:46.561031 9320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:19:46.562994 9320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:19:46.579599 9333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:19:46.581290 9320 server_base.cc:1034] running on GCE node
W20260808 02:19:46.581039 9337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:19:46.582100 9332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:19:46.582870 9320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:19:46.583942 9320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:19:46.587248 9320 hybrid_clock.cc:648] HybridClock initialized: now 1786155586587136 us; error 121 us; skew 500 ppm
Aug 08 02:19:46 dist-test-slave-1vmg krb5kdc[8281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155586, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:19:46.591856 9320 init.cc:377] Logged in from keytab as kudu/127.24.79.194@KRBTEST.COM (short username kudu)
I20260808 02:19:46.593371 9320 webserver.cc:460] Webserver started at http://127.24.79.194:39329/ using document root <none> and password file <none>
I20260808 02:19:46.594322 9320 fs_manager.cc:359] Metadata directory not provided
I20260808 02:19:46.594517 9320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:19:46.594897 9320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:19:46 dist-test-slave-1vmg krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155586, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for HTTP/127.24.79.212@KRBTEST.COM
I20260808 02:19:46.625694 9320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "761de31f62a64a69b1356eb81225b6e1"
format_stamp: "Formatted at 2026-08-08 02:19:46 on dist-test-slave-1vmg"
server_key: "a43ab8b33ae7c047be0946e126f968a3"
server_key_iv: "ce3de293394d31ccca2ee6a21eef31fe"
server_key_version: "kuduclusterkey@0"
I20260808 02:19:46.626775 9320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "761de31f62a64a69b1356eb81225b6e1"
format_stamp: "Formatted at 2026-08-08 02:19:46 on dist-test-slave-1vmg"
server_key: "a43ab8b33ae7c047be0946e126f968a3"
server_key_iv: "ce3de293394d31ccca2ee6a21eef31fe"
server_key_version: "kuduclusterkey@0"
I20260808 02:19:46.640581 9320 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.005s sys 0.000s
I20260808 02:19:46.669461 9347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:19:46.674269 9320 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.004s sys 0.000s
I20260808 02:19:46.674485 9320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "761de31f62a64a69b1356eb81225b6e1"
format_stamp: "Formatted at 2026-08-08 02:19:46 on dist-test-slave-1vmg"
server_key: "a43ab8b33ae7c047be0946e126f968a3"
server_key_iv: "ce3de293394d31ccca2ee6a21eef31fe"
server_key_version: "kuduclusterkey@0"
I20260808 02:19:46.674666 9320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:19:46.726580 9320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:19:46.731451 9320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:19:46.731792 9320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:19:46.732770 9320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:19:46.734117 9320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:19:46.734243 9320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260808 02:19:46.734354 9320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:19:46.734418 9320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:19:46.747476 9320 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:37475
I20260808 02:19:46.748504 9516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:37475 every 8 connection(s)
I20260808 02:19:46.749101 9320 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:19:46.751475 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 9320
I20260808 02:19:46.751629 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:19:46.751971 24895 external_mini_cluster.cc:1428] Setting key 8e10929910cdea6d94236ccb0cd34289
Aug 08 02:19:46 dist-test-slave-1vmg krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155586, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:19:46.775007 9314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:19:46.754472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:48771 (local address 127.24.79.254:41773)
0808 02:19:46.755294 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0808 02:19:46.755299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:19:46.755319 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:19:46.755807 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:19:46.755811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:19:46.755868 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:19:46.755926 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:19:46.757969 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:19:46.758881 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:19:46.761574 (+ 2693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:19:46.761972 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:19:46.762086 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:19:46.769420 (+ 7334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:19:46.769425 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:19:46.769428 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:19:46.769468 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:19:46.771689 (+ 2221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:19:46.772570 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:19:46.772575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:19:46.772577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:19:46.772700 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:19:46.773252 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:19:46.773256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:19:46.773258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:19:46.773537 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:19:46.773645 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:19:46.774607 (+ 962us) server_negotiation.cc:299] Negotiation successful
0808 02:19:46.774786 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733}
I20260808 02:19:46.776176 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:19:46.752870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:41773 (local address 127.24.79.194:48771)
0808 02:19:46.754091 (+ 1221us) negotiation.cc:107] Waiting for socket to connect
0808 02:19:46.754196 (+ 105us) client_negotiation.cc:174] Beginning negotiation
0808 02:19:46.755483 (+ 1287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:19:46.756496 (+ 1013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:19:46.756507 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:19:46.756949 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:19:46.757760 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:19:46.757785 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:19:46.759124 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:19:46.759128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:19:46.761344 (+ 2216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:19:46.761366 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:19:46.765769 (+ 4403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:19:46.765774 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:19:46.766157 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:19:46.767158 (+ 1001us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:19:46.767186 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:19:46.769118 (+ 1932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:19:46.771906 (+ 2788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:19:46.771916 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:19:46.771943 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:19:46.772428 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:19:46.772838 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:19:46.772841 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:19:46.772843 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:19:46.773126 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:19:46.773674 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:19:46.773683 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:19:46.774126 (+ 443us) client_negotiation.cc:769] Sending connection context
0808 02:19:46.774394 (+ 268us) client_negotiation.cc:240] Negotiation successful
0808 02:19:46.774864 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":929,"thread_start_us":219,"threads_started":1}
I20260808 02:19:46.777884 9520 heartbeater.cc:347] Connected to a master server at 127.24.79.254:41773
I20260808 02:19:46.778210 9520 heartbeater.cc:464] Registering TS with master...
I20260808 02:19:46.778986 9520 heartbeater.cc:511] Master 127.24.79.254:41773 requested a full tablet report, sending...
I20260808 02:19:46.780599 8995 ts_manager.cc:194] Registered new tserver with Master: 761de31f62a64a69b1356eb81225b6e1 (127.24.79.194:37475)
I20260808 02:19:46.781548 8995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.194@KRBTEST.COM'} at 127.24.79.194:48771
WARNING: no policy specified for kudu/127.24.79.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
I20260808 02:19:46.830219 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.79.212:47441/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:41773
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:36265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:19:46 dist-test-slave-1vmg krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155586, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for HTTP/127.24.79.212@KRBTEST.COM
W20260808 02:19:47.072710 9526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:19:47.073258 9526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:19:47.073372 9526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:19:47.079834 9526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:19:47.087316 9526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:19:47.087492 9526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:19:47.106464 9526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:36265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.79.212:47441/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:41773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:19:47.107893 9526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:19:47.109105 9526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:19:47.125144 9542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:19:47.125733 9537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:19:47.125994 9544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:19:47.127863 9526 server_base.cc:1034] running on GCE node
I20260808 02:19:47.128576 9526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:19:47.129633 9526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:19:47.133072 9526 hybrid_clock.cc:648] HybridClock initialized: now 1786155587132964 us; error 110 us; skew 500 ppm
Aug 08 02:19:47 dist-test-slave-1vmg krb5kdc[8281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155587, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:19:47.137913 9526 init.cc:377] Logged in from keytab as kudu/127.24.79.195@KRBTEST.COM (short username kudu)
I20260808 02:19:47.139489 9526 webserver.cc:460] Webserver started at http://127.24.79.195:41107/ using document root <none> and password file <none>
I20260808 02:19:47.140640 9526 fs_manager.cc:359] Metadata directory not provided
I20260808 02:19:47.140868 9526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:19:47.141459 9526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:19:47 dist-test-slave-1vmg krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155587, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for HTTP/127.24.79.212@KRBTEST.COM
I20260808 02:19:47.169266 9526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "bcec34a4a6554dbb84a1b72f91a488fb"
format_stamp: "Formatted at 2026-08-08 02:19:47 on dist-test-slave-1vmg"
server_key: "9e39a39c0991ad22c69da2d6fb320d1a"
server_key_iv: "4717f2f8a23c5615b7cba629231bf12f"
server_key_version: "kuduclusterkey@0"
I20260808 02:19:47.172281 9526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "bcec34a4a6554dbb84a1b72f91a488fb"
format_stamp: "Formatted at 2026-08-08 02:19:47 on dist-test-slave-1vmg"
server_key: "9e39a39c0991ad22c69da2d6fb320d1a"
server_key_iv: "4717f2f8a23c5615b7cba629231bf12f"
server_key_version: "kuduclusterkey@0"
I20260808 02:19:47.179885 9526 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260808 02:19:47.212785 9556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:19:47.214779 9526 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260808 02:19:47.215009 9526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "bcec34a4a6554dbb84a1b72f91a488fb"
format_stamp: "Formatted at 2026-08-08 02:19:47 on dist-test-slave-1vmg"
server_key: "9e39a39c0991ad22c69da2d6fb320d1a"
server_key_iv: "4717f2f8a23c5615b7cba629231bf12f"
server_key_version: "kuduclusterkey@0"
I20260808 02:19:47.215196 9526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:19:47.250387 9526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:19:47.254750 9526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:19:47.255072 9526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:19:47.255899 9526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:19:47.257210 9526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:19:47.257326 9526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:19:47.257541 9526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:19:47.257666 9526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:19:47.277467 9526 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:32899
I20260808 02:19:47.279346 9526 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:19:47.282087 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 9526
I20260808 02:19:47.282315 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:19:47.282717 24895 external_mini_cluster.cc:1428] Setting key b41389b623bb8708ecb788fcd1182730
I20260808 02:19:47.284113 9719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:32899 every 8 connection(s)
I20260808 02:19:47.287946 9310 heartbeater.cc:503] Master 127.24.79.254:41773 was elected leader, sending a full tablet report...
Aug 08 02:19:47 dist-test-slave-1vmg krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155587, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:19:47.308109 9722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:19:47.287130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:50235 (local address 127.24.79.254:41773)
0808 02:19:47.290781 (+ 3651us) server_negotiation.cc:206] Beginning negotiation
0808 02:19:47.290786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:19:47.290807 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:19:47.291070 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:19:47.291074 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:19:47.291138 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:19:47.291262 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:19:47.292897 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.293726 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.295970 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.296379 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.296539 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:19:47.302136 (+ 5597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:19:47.302141 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:19:47.302144 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:19:47.302180 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:19:47.304258 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:19:47.305250 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:19:47.305257 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:19:47.305259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:19:47.305402 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:19:47.306101 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:19:47.306106 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:19:47.306107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:19:47.306428 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:19:47.306553 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 02:19:47.307598 (+ 1045us) server_negotiation.cc:299] Negotiation successful
0808 02:19:47.307907 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3553,"thread_start_us":130,"threads_started":1}
I20260808 02:19:47.309146 9723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:19:47.287744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:41773 (local address 127.24.79.195:50235)
0808 02:19:47.289571 (+ 1827us) negotiation.cc:107] Waiting for socket to connect
0808 02:19:47.289652 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0808 02:19:47.290771 (+ 1119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:19:47.291475 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:19:47.291485 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:19:47.291881 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:19:47.292669 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:19:47.292690 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.293915 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.293919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:19:47.295517 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:19:47.295536 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.296603 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.296607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:19:47.296855 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:19:47.297701 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:19:47.297726 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:19:47.301855 (+ 4129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:19:47.304460 (+ 2605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:19:47.304471 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:19:47.304498 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:19:47.304964 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:19:47.305576 (+ 612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:19:47.305581 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:19:47.305583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:19:47.305909 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:19:47.306591 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:19:47.306601 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:19:47.307131 (+ 530us) client_negotiation.cc:769] Sending connection context
0808 02:19:47.307408 (+ 277us) client_negotiation.cc:240] Negotiation successful
0808 02:19:47.307870 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1537,"thread_start_us":183,"threads_started":1}
I20260808 02:19:47.311225 9720 heartbeater.cc:347] Connected to a master server at 127.24.79.254:41773
I20260808 02:19:47.311559 9720 heartbeater.cc:464] Registering TS with master...
I20260808 02:19:47.312282 9720 heartbeater.cc:511] Master 127.24.79.254:41773 requested a full tablet report, sending...
I20260808 02:19:47.313773 8995 ts_manager.cc:194] Registered new tserver with Master: bcec34a4a6554dbb84a1b72f91a488fb (127.24.79.195:32899)
I20260808 02:19:47.314498 8995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.195@KRBTEST.COM'} at 127.24.79.195:50235
I20260808 02:19:47.317329 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 02:19:47 dist-test-slave-1vmg krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155586, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for HTTP/127.24.79.212@KRBTEST.COM
I20260808 02:19:47.339720 9722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:19:47.321038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35452 (local address 127.24.79.254:41773)
0808 02:19:47.321191 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 02:19:47.321195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:19:47.321989 (+ 794us) server_negotiation.cc:407] Connection header received
0808 02:19:47.322205 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:19:47.322209 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:19:47.322261 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:19:47.322313 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:19:47.323584 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.324401 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.325615 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.325962 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.326040 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:19:47.328978 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:19:47.329006 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:19:47.329009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:19:47.329050 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:19:47.331952 (+ 2902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:19:47.337142 (+ 5190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:19:47.337148 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:19:47.337150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:19:47.337311 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:19:47.338460 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:19:47.338465 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:19:47.338467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:19:47.338816 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:19:47.339023 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 02:19:47.339349 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 02:19:47.339523 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 02:19:47.343786 8995 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35452:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:19:47.348872 8995 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:19:47.376919 9740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:19:47.369394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:41853 (local address 127.0.0.1:60840)
0808 02:19:47.369847 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 02:19:47.369993 (+ 146us) client_negotiation.cc:174] Beginning negotiation
0808 02:19:47.370155 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:19:47.371655 (+ 1500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:19:47.371661 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:19:47.371689 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:19:47.371841 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:19:47.371848 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.373439 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.373443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:19:47.374887 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:19:47.374898 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.376330 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.376334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:19:47.376462 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:19:47.376504 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 02:19:47.376623 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 02:19:47.376759 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":134,"threads_started":1}
I20260808 02:19:47.377406 9743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:19:47.370521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60840 (local address 127.24.79.193:41853)
0808 02:19:47.371104 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0808 02:19:47.371112 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:19:47.371139 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:19:47.371224 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:19:47.371230 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:19:47.371471 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:19:47.371631 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:19:47.371986 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.373285 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.375061 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.375947 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.376059 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:19:47.376257 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 02:19:47.377148 (+ 891us) server_negotiation.cc:299] Negotiation successful
0808 02:19:47.377269 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":159,"threads_started":1}
I20260808 02:19:47.380985 9189 tablet_service.cc:1467] Processing CreateTablet for tablet e555a5590fd84b97b7258ee98007374e (DEFAULT_TABLE table=test-table [id=f6afdd8933994c8cbce41e2aab010f49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:19:47.383970 9189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e555a5590fd84b97b7258ee98007374e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:19:47.384055 9741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:19:47.372982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:32899 (local address 127.0.0.1:57838)
0808 02:19:47.375102 (+ 2120us) negotiation.cc:107] Waiting for socket to connect
0808 02:19:47.375126 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 02:19:47.375227 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:19:47.377652 (+ 2425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:19:47.377655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:19:47.377676 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:19:47.377769 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:19:47.377774 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.380049 (+ 2275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.380053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:19:47.381201 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:19:47.381215 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.383565 (+ 2350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.383570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:19:47.383688 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:19:47.383722 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 02:19:47.383794 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 02:19:47.383891 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2037,"thread_start_us":882,"threads_started":1}
I20260808 02:19:47.385566 9742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:19:47.370501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39570 (local address 127.24.79.194:37475)
0808 02:19:47.371966 (+ 1465us) server_negotiation.cc:206] Beginning negotiation
0808 02:19:47.371972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:19:47.374129 (+ 2157us) server_negotiation.cc:407] Connection header received
0808 02:19:47.374362 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:19:47.374369 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:19:47.374606 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:19:47.374749 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:19:47.375529 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.379260 (+ 3731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.380945 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.383148 (+ 2203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.383917 (+ 769us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:19:47.384097 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 02:19:47.384519 (+ 422us) server_negotiation.cc:299] Negotiation successful
0808 02:19:47.384863 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1348,"thread_start_us":116,"threads_started":1}
I20260808 02:19:47.386220 9746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:19:47.373472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57838 (local address 127.24.79.195:32899)
0808 02:19:47.375152 (+ 1680us) server_negotiation.cc:206] Beginning negotiation
0808 02:19:47.375165 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:19:47.375580 (+ 415us) server_negotiation.cc:407] Connection header received
0808 02:19:47.375683 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:19:47.375688 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:19:47.377365 (+ 1677us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:19:47.378328 (+ 963us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:19:47.378489 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.379645 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.381806 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.383290 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.385128 (+ 1838us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:19:47.385306 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 02:19:47.385360 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 02:19:47.385457 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1567,"thread_start_us":192,"threads_started":1}
I20260808 02:19:47.386030 9748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:19:47.369405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:37475 (local address 127.0.0.1:39570)
0808 02:19:47.374008 (+ 4603us) negotiation.cc:107] Waiting for socket to connect
0808 02:19:47.374023 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:19:47.374213 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:19:47.375265 (+ 1052us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:19:47.375268 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:19:47.375291 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:19:47.375392 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:19:47.375397 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.379418 (+ 4021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.379422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:19:47.380737 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:19:47.380749 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.384212 (+ 3463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.384216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:19:47.384313 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:19:47.384336 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:19:47.385802 (+ 1466us) client_negotiation.cc:240] Negotiation successful
0808 02:19:47.385895 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4531,"mutex_wait_us":2,"thread_start_us":148,"threads_started":1}
I20260808 02:19:47.389895 9620 tablet_service.cc:1467] Processing CreateTablet for tablet e555a5590fd84b97b7258ee98007374e (DEFAULT_TABLE table=test-table [id=f6afdd8933994c8cbce41e2aab010f49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:19:47.389899 9417 tablet_service.cc:1467] Processing CreateTablet for tablet e555a5590fd84b97b7258ee98007374e (DEFAULT_TABLE table=test-table [id=f6afdd8933994c8cbce41e2aab010f49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:19:47.391342 9620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e555a5590fd84b97b7258ee98007374e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:19:47.391343 9417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e555a5590fd84b97b7258ee98007374e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:19:47.401724 9751 tablet_bootstrap.cc:492] T e555a5590fd84b97b7258ee98007374e P bcec34a4a6554dbb84a1b72f91a488fb: Bootstrap starting.
I20260808 02:19:47.404644 9753 tablet_bootstrap.cc:492] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc: Bootstrap starting.
I20260808 02:19:47.406512 9751 tablet_bootstrap.cc:654] T e555a5590fd84b97b7258ee98007374e P bcec34a4a6554dbb84a1b72f91a488fb: Neither blocks nor log segments found. Creating new log.
I20260808 02:19:47.407850 9751 log.cc:824] T e555a5590fd84b97b7258ee98007374e P bcec34a4a6554dbb84a1b72f91a488fb: Log is configured to *not* fsync() on all Append() calls
I20260808 02:19:47.411491 9753 tablet_bootstrap.cc:654] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc: Neither blocks nor log segments found. Creating new log.
I20260808 02:19:47.414305 9753 log.cc:824] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc: Log is configured to *not* fsync() on all Append() calls
I20260808 02:19:47.424913 9751 tablet_bootstrap.cc:492] T e555a5590fd84b97b7258ee98007374e P bcec34a4a6554dbb84a1b72f91a488fb: No bootstrap required, opened a new log
I20260808 02:19:47.425340 9751 ts_tablet_manager.cc:1397] T e555a5590fd84b97b7258ee98007374e P bcec34a4a6554dbb84a1b72f91a488fb: Time spent bootstrapping tablet: real 0.024s user 0.008s sys 0.000s
I20260808 02:19:47.426765 9755 tablet_bootstrap.cc:492] T e555a5590fd84b97b7258ee98007374e P 761de31f62a64a69b1356eb81225b6e1: Bootstrap starting.
I20260808 02:19:47.433495 9755 tablet_bootstrap.cc:654] T e555a5590fd84b97b7258ee98007374e P 761de31f62a64a69b1356eb81225b6e1: Neither blocks nor log segments found. Creating new log.
I20260808 02:19:47.434687 9755 log.cc:824] T e555a5590fd84b97b7258ee98007374e P 761de31f62a64a69b1356eb81225b6e1: Log is configured to *not* fsync() on all Append() calls
I20260808 02:19:47.445837 9755 tablet_bootstrap.cc:492] T e555a5590fd84b97b7258ee98007374e P 761de31f62a64a69b1356eb81225b6e1: No bootstrap required, opened a new log
I20260808 02:19:47.446501 9753 tablet_bootstrap.cc:492] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc: No bootstrap required, opened a new log
I20260808 02:19:47.448163 9753 ts_tablet_manager.cc:1397] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc: Time spent bootstrapping tablet: real 0.044s user 0.005s sys 0.003s
I20260808 02:19:47.451314 9751 raft_consensus.cc:348] T e555a5590fd84b97b7258ee98007374e P bcec34a4a6554dbb84a1b72f91a488fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a71fe1c0e2a14fedb195d3b91244fcdc" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41853 } } peers { permanent_uuid: "761de31f62a64a69b1356eb81225b6e1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37475 } } peers { permanent_uuid: "bcec34a4a6554dbb84a1b72f91a488fb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 32899 } }
I20260808 02:19:47.454171 9751 raft_consensus.cc:374] T e555a5590fd84b97b7258ee98007374e P bcec34a4a6554dbb84a1b72f91a488fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:19:47.454429 9751 raft_consensus.cc:729] T e555a5590fd84b97b7258ee98007374e P bcec34a4a6554dbb84a1b72f91a488fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcec34a4a6554dbb84a1b72f91a488fb, State: Initialized, Role: FOLLOWER
I20260808 02:19:47.456370 9753 raft_consensus.cc:348] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a71fe1c0e2a14fedb195d3b91244fcdc" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41853 } } peers { permanent_uuid: "761de31f62a64a69b1356eb81225b6e1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37475 } } peers { permanent_uuid: "bcec34a4a6554dbb84a1b72f91a488fb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 32899 } }
I20260808 02:19:47.457465 9753 raft_consensus.cc:374] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:19:47.457602 9753 raft_consensus.cc:729] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a71fe1c0e2a14fedb195d3b91244fcdc, State: Initialized, Role: FOLLOWER
I20260808 02:19:47.458060 9751 consensus_queue.cc:260] T e555a5590fd84b97b7258ee98007374e P bcec34a4a6554dbb84a1b72f91a488fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a71fe1c0e2a14fedb195d3b91244fcdc" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41853 } } peers { permanent_uuid: "761de31f62a64a69b1356eb81225b6e1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37475 } } peers { permanent_uuid: "bcec34a4a6554dbb84a1b72f91a488fb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 32899 } }
I20260808 02:19:47.458447 9753 consensus_queue.cc:260] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a71fe1c0e2a14fedb195d3b91244fcdc" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41853 } } peers { permanent_uuid: "761de31f62a64a69b1356eb81225b6e1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37475 } } peers { permanent_uuid: "bcec34a4a6554dbb84a1b72f91a488fb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 32899 } }
I20260808 02:19:47.459273 9755 ts_tablet_manager.cc:1397] T e555a5590fd84b97b7258ee98007374e P 761de31f62a64a69b1356eb81225b6e1: Time spent bootstrapping tablet: real 0.039s user 0.008s sys 0.000s
I20260808 02:19:47.460323 9753 ts_tablet_manager.cc:1428] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc: Time spent starting tablet: real 0.012s user 0.003s sys 0.005s
I20260808 02:19:47.464560 9755 raft_consensus.cc:348] T e555a5590fd84b97b7258ee98007374e P 761de31f62a64a69b1356eb81225b6e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a71fe1c0e2a14fedb195d3b91244fcdc" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41853 } } peers { permanent_uuid: "761de31f62a64a69b1356eb81225b6e1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37475 } } peers { permanent_uuid: "bcec34a4a6554dbb84a1b72f91a488fb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 32899 } }
I20260808 02:19:47.465210 9755 raft_consensus.cc:374] T e555a5590fd84b97b7258ee98007374e P 761de31f62a64a69b1356eb81225b6e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:19:47.465366 9755 raft_consensus.cc:729] T e555a5590fd84b97b7258ee98007374e P 761de31f62a64a69b1356eb81225b6e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 761de31f62a64a69b1356eb81225b6e1, State: Initialized, Role: FOLLOWER
I20260808 02:19:47.465497 9751 ts_tablet_manager.cc:1428] T e555a5590fd84b97b7258ee98007374e P bcec34a4a6554dbb84a1b72f91a488fb: Time spent starting tablet: real 0.040s user 0.002s sys 0.005s
I20260808 02:19:47.466060 9755 consensus_queue.cc:260] T e555a5590fd84b97b7258ee98007374e P 761de31f62a64a69b1356eb81225b6e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a71fe1c0e2a14fedb195d3b91244fcdc" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41853 } } peers { permanent_uuid: "761de31f62a64a69b1356eb81225b6e1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37475 } } peers { permanent_uuid: "bcec34a4a6554dbb84a1b72f91a488fb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 32899 } }
I20260808 02:19:47.466773 9720 heartbeater.cc:503] Master 127.24.79.254:41773 was elected leader, sending a full tablet report...
I20260808 02:19:47.467521 9755 ts_tablet_manager.cc:1428] T e555a5590fd84b97b7258ee98007374e P 761de31f62a64a69b1356eb81225b6e1: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260808 02:19:47.469364 9520 heartbeater.cc:503] Master 127.24.79.254:41773 was elected leader, sending a full tablet report...
W20260808 02:19:47.485838 9311 tablet.cc:2369] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:19:47.501125 9521 tablet.cc:2369] T e555a5590fd84b97b7258ee98007374e P 761de31f62a64a69b1356eb81225b6e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:19:47.536972 9721 tablet.cc:2369] T e555a5590fd84b97b7258ee98007374e P bcec34a4a6554dbb84a1b72f91a488fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:19:47.612577 9759 raft_consensus.cc:482] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:19:47.612880 9759 raft_consensus.cc:504] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a71fe1c0e2a14fedb195d3b91244fcdc" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41853 } } peers { permanent_uuid: "761de31f62a64a69b1356eb81225b6e1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37475 } } peers { permanent_uuid: "bcec34a4a6554dbb84a1b72f91a488fb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 32899 } }
I20260808 02:19:47.614534 9759 leader_election.cc:288] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 761de31f62a64a69b1356eb81225b6e1 (127.24.79.194:37475), bcec34a4a6554dbb84a1b72f91a488fb (127.24.79.195:32899)
I20260808 02:19:47.623991 9742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:19:47.615358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:58343 (local address 127.24.79.194:37475)
0808 02:19:47.615586 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 02:19:47.615592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:19:47.618517 (+ 2925us) server_negotiation.cc:407] Connection header received
0808 02:19:47.618766 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:19:47.618771 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:19:47.618824 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:19:47.618880 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:19:47.619413 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.620212 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.621964 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.622581 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.622689 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:19:47.622749 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 02:19:47.623724 (+ 975us) server_negotiation.cc:299] Negotiation successful
0808 02:19:47.623831 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 02:19:47.624742 9764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:19:47.615527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:37475 (local address 127.24.79.193:58343)
0808 02:19:47.618385 (+ 2858us) negotiation.cc:107] Waiting for socket to connect
0808 02:19:47.618403 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:19:47.618626 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:19:47.619074 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:19:47.619077 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:19:47.619097 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:19:47.619265 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:19:47.619272 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.620383 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.620387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:19:47.621793 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:19:47.621808 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.622891 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.622895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:19:47.623545 (+ 650us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:19:47.623577 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 02:19:47.624473 (+ 896us) client_negotiation.cc:240] Negotiation successful
0808 02:19:47.624568 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2764,"thread_start_us":177,"threads_started":1}
I20260808 02:19:47.625908 9444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e555a5590fd84b97b7258ee98007374e" candidate_uuid: "a71fe1c0e2a14fedb195d3b91244fcdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "761de31f62a64a69b1356eb81225b6e1" is_pre_election: true
I20260808 02:19:47.626312 9444 raft_consensus.cc:2454] T e555a5590fd84b97b7258ee98007374e P 761de31f62a64a69b1356eb81225b6e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a71fe1c0e2a14fedb195d3b91244fcdc in term 0.
I20260808 02:19:47.627029 9146 leader_election.cc:302] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 761de31f62a64a69b1356eb81225b6e1, a71fe1c0e2a14fedb195d3b91244fcdc; no voters:
I20260808 02:19:47.627454 9759 raft_consensus.cc:2781] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:19:47.627559 9759 raft_consensus.cc:482] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:19:47.627619 9759 raft_consensus.cc:3037] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:19:47.629210 9759 raft_consensus.cc:504] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a71fe1c0e2a14fedb195d3b91244fcdc" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41853 } } peers { permanent_uuid: "761de31f62a64a69b1356eb81225b6e1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37475 } } peers { permanent_uuid: "bcec34a4a6554dbb84a1b72f91a488fb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 32899 } }
I20260808 02:19:47.629705 9759 leader_election.cc:288] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc [CANDIDATE]: Term 1 election: Requested vote from peers 761de31f62a64a69b1356eb81225b6e1 (127.24.79.194:37475), bcec34a4a6554dbb84a1b72f91a488fb (127.24.79.195:32899)
I20260808 02:19:47.630559 9444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e555a5590fd84b97b7258ee98007374e" candidate_uuid: "a71fe1c0e2a14fedb195d3b91244fcdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "761de31f62a64a69b1356eb81225b6e1"
I20260808 02:19:47.630741 9444 raft_consensus.cc:3037] T e555a5590fd84b97b7258ee98007374e P 761de31f62a64a69b1356eb81225b6e1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:19:47.632414 9444 raft_consensus.cc:2454] T e555a5590fd84b97b7258ee98007374e P 761de31f62a64a69b1356eb81225b6e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a71fe1c0e2a14fedb195d3b91244fcdc in term 1.
I20260808 02:19:47.633069 9146 leader_election.cc:302] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 761de31f62a64a69b1356eb81225b6e1, a71fe1c0e2a14fedb195d3b91244fcdc; no voters:
I20260808 02:19:47.633402 9746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:19:47.616534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:48021 (local address 127.24.79.195:32899)
0808 02:19:47.616695 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 02:19:47.616700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:19:47.621621 (+ 4921us) server_negotiation.cc:407] Connection header received
0808 02:19:47.622664 (+ 1043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:19:47.622668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:19:47.622725 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:19:47.622795 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:19:47.629187 (+ 6392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.630096 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.631908 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.632577 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.632690 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:19:47.632750 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 02:19:47.633161 (+ 411us) server_negotiation.cc:299] Negotiation successful
0808 02:19:47.633263 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 02:19:47.633622 9759 raft_consensus.cc:2781] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:19:47.633895 9759 raft_consensus.cc:686] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc [term 1 LEADER]: Becoming Leader. State: Replica: a71fe1c0e2a14fedb195d3b91244fcdc, State: Running, Role: LEADER
I20260808 02:19:47.633971 9765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:19:47.616068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:32899 (local address 127.24.79.193:48021)
0808 02:19:47.621517 (+ 5449us) negotiation.cc:107] Waiting for socket to connect
0808 02:19:47.621533 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:19:47.622507 (+ 974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:19:47.623001 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:19:47.623004 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:19:47.623026 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:19:47.628935 (+ 5909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:19:47.628958 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.630381 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.630385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:19:47.631714 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:19:47.631730 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.632867 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.632870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:19:47.632986 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:19:47.633012 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 02:19:47.633695 (+ 683us) client_negotiation.cc:240] Negotiation successful
0808 02:19:47.633794 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5364,"mutex_wait_us":5741,"thread_start_us":127,"threads_started":1}
I20260808 02:19:47.634333 9759 consensus_queue.cc:237] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a71fe1c0e2a14fedb195d3b91244fcdc" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41853 } } peers { permanent_uuid: "761de31f62a64a69b1356eb81225b6e1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37475 } } peers { permanent_uuid: "bcec34a4a6554dbb84a1b72f91a488fb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 32899 } }
I20260808 02:19:47.635982 9626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e555a5590fd84b97b7258ee98007374e" candidate_uuid: "a71fe1c0e2a14fedb195d3b91244fcdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcec34a4a6554dbb84a1b72f91a488fb" is_pre_election: true
I20260808 02:19:47.636382 9626 raft_consensus.cc:2454] T e555a5590fd84b97b7258ee98007374e P bcec34a4a6554dbb84a1b72f91a488fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a71fe1c0e2a14fedb195d3b91244fcdc in term 0.
I20260808 02:19:47.636821 9649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e555a5590fd84b97b7258ee98007374e" candidate_uuid: "a71fe1c0e2a14fedb195d3b91244fcdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcec34a4a6554dbb84a1b72f91a488fb"
I20260808 02:19:47.636952 9649 raft_consensus.cc:3037] T e555a5590fd84b97b7258ee98007374e P bcec34a4a6554dbb84a1b72f91a488fb [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:19:47.638669 9649 raft_consensus.cc:2454] T e555a5590fd84b97b7258ee98007374e P bcec34a4a6554dbb84a1b72f91a488fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a71fe1c0e2a14fedb195d3b91244fcdc in term 1.
I20260808 02:19:47.639926 8995 catalog_manager.cc:5495] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc reported cstate change: term changed from 0 to 1, leader changed from <none> to a71fe1c0e2a14fedb195d3b91244fcdc (127.24.79.193). New cstate: current_term: 1 leader_uuid: "a71fe1c0e2a14fedb195d3b91244fcdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a71fe1c0e2a14fedb195d3b91244fcdc" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "761de31f62a64a69b1356eb81225b6e1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcec34a4a6554dbb84a1b72f91a488fb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 32899 } health_report { overall_health: UNKNOWN } } }
I20260808 02:19:47.689455 9743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:19:47.685168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60848 (local address 127.24.79.193:41853)
0808 02:19:47.685519 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0808 02:19:47.685522 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:19:47.685534 (+ 12us) server_negotiation.cc:407] Connection header received
0808 02:19:47.685631 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:19:47.685634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:19:47.685690 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:19:47.685791 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:19:47.686068 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.686581 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.687550 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:19:47.687941 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:19:47.688073 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:19:47.688427 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:19:47.688923 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:19:47.689044 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 02:19:47.689188 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 02:19:47.689268 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265}
I20260808 02:19:47.696552 9444 raft_consensus.cc:1261] T e555a5590fd84b97b7258ee98007374e P 761de31f62a64a69b1356eb81225b6e1 [term 1 FOLLOWER]: Refusing update from remote peer a71fe1c0e2a14fedb195d3b91244fcdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:19:47.696789 9649 raft_consensus.cc:1261] T e555a5590fd84b97b7258ee98007374e P bcec34a4a6554dbb84a1b72f91a488fb [term 1 FOLLOWER]: Refusing update from remote peer a71fe1c0e2a14fedb195d3b91244fcdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:19:47.697501 9766 consensus_queue.cc:1035] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "761de31f62a64a69b1356eb81225b6e1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:19:47.697726 9759 consensus_queue.cc:1035] T e555a5590fd84b97b7258ee98007374e P a71fe1c0e2a14fedb195d3b91244fcdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcec34a4a6554dbb84a1b72f91a488fb" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:19:47.708331 9781 mvcc.cc:205] Tried to move back new op lower bound from 7316093287195918336 to 7316093286958321664. Current Snapshot: MvccSnapshot[applied={T|T < 7316093287195918336}]
I20260808 02:19:47.714509 9784 mvcc.cc:205] Tried to move back new op lower bound from 7316093287195918336 to 7316093286958321664. Current Snapshot: MvccSnapshot[applied={T|T < 7316093287195918336}]
I20260808 02:19:47.734164 8995 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:19:47.734381 8995 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:19:47.737955 8995 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50b9f424434744f9b531ca6b47aa5ca4: Sending DeleteTablet for 3 replicas of tablet e555a5590fd84b97b7258ee98007374e
I20260808 02:19:47.739277 9189 tablet_service.cc:1514] Processing DeleteTablet for tablet e555a5590fd84b97b7258ee98007374e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:19:47 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:60840
I20260808 02:19:47.740837 9417 tablet_service.cc:1514] Processing DeleteTablet for tablet e555a5590fd84b97b7258ee98007374e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:19:47 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:39570
I20260808 02:19:47.740852 9620 tablet_service.cc:1514] Processing DeleteTablet for tablet e555a5590fd84b97b7258ee98007374e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:19:47 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:57838
I20260808 02:19:47.742401 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 9119
I20260808 02:19:47.750471 9798 tablet_replica.cc:330] T e555a5590fd84b97b7258ee98007374e P bcec34a4a6554dbb84a1b72f91a488fb: stopping tablet replica
I20260808 02:19:47.751060 9798 raft_consensus.cc:2229] T e555a5590fd84b97b7258ee98007374e P bcec34a4a6554dbb84a1b72f91a488fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:19:47.751569 9798 raft_consensus.cc:2258] T e555a5590fd84b97b7258ee98007374e P bcec34a4a6554dbb84a1b72f91a488fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:19:47.754900 9798 ts_tablet_manager.cc:1905] T e555a5590fd84b97b7258ee98007374e P bcec34a4a6554dbb84a1b72f91a488fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:19:47.755699 9796 tablet_replica.cc:330] T e555a5590fd84b97b7258ee98007374e P 761de31f62a64a69b1356eb81225b6e1: stopping tablet replica
I20260808 02:19:47.756178 9796 raft_consensus.cc:2229] T e555a5590fd84b97b7258ee98007374e P 761de31f62a64a69b1356eb81225b6e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:19:47.756743 9796 raft_consensus.cc:2258] T e555a5590fd84b97b7258ee98007374e P 761de31f62a64a69b1356eb81225b6e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:19:47.758502 9796 ts_tablet_manager.cc:1905] T e555a5590fd84b97b7258ee98007374e P 761de31f62a64a69b1356eb81225b6e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:19:47.758710 9798 ts_tablet_manager.cc:1918] T e555a5590fd84b97b7258ee98007374e P bcec34a4a6554dbb84a1b72f91a488fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:19:47.758880 9798 log.cc:1196] T e555a5590fd84b97b7258ee98007374e P bcec34a4a6554dbb84a1b72f91a488fb: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/e555a5590fd84b97b7258ee98007374e
I20260808 02:19:47.759366 9798 ts_tablet_manager.cc:1939] T e555a5590fd84b97b7258ee98007374e P bcec34a4a6554dbb84a1b72f91a488fb: Deleting consensus metadata
I20260808 02:19:47.762501 9796 ts_tablet_manager.cc:1918] T e555a5590fd84b97b7258ee98007374e P 761de31f62a64a69b1356eb81225b6e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:19:47.762835 9796 log.cc:1196] T e555a5590fd84b97b7258ee98007374e P 761de31f62a64a69b1356eb81225b6e1: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/e555a5590fd84b97b7258ee98007374e
I20260808 02:19:47.763319 9796 ts_tablet_manager.cc:1939] T e555a5590fd84b97b7258ee98007374e P 761de31f62a64a69b1356eb81225b6e1: Deleting consensus metadata
I20260808 02:19:47.765312 8981 catalog_manager.cc:4841] TS 761de31f62a64a69b1356eb81225b6e1 (127.24.79.194:37475): tablet e555a5590fd84b97b7258ee98007374e (table test-table [id=f6afdd8933994c8cbce41e2aab010f49]) successfully deleted
I20260808 02:19:47.772735 8981 catalog_manager.cc:4841] TS bcec34a4a6554dbb84a1b72f91a488fb (127.24.79.195:32899): tablet e555a5590fd84b97b7258ee98007374e (table test-table [id=f6afdd8933994c8cbce41e2aab010f49]) successfully deleted
W20260808 02:19:47.779819 8980 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.79.193:41853 (error 108)
I20260808 02:19:47.780364 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 9320
I20260808 02:19:47.785868 9748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:19:47.785491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:41853 (local address 127.0.0.1:60860)
0808 02:19:47.785651 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 02:19:47.785751 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.193:41853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 02:19:47.786103 8980 catalog_manager.cc:4584] TS a71fe1c0e2a14fedb195d3b91244fcdc (127.24.79.193:41853): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e555a5590fd84b97b7258ee98007374e: Network error: Client connection negotiation failed: client connection to 127.24.79.193:41853: connect: Connection refused (error 111)
I20260808 02:19:47.796512 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 9526
I20260808 02:19:47.821195 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 8955
I20260808 02:19:47.839887 24895 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 02:19:48.191763 24895 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 02:19:48.191920 24895 mini_ranger.cc:66] Stopping Ranger...
I20260808 02:19:48.540752 24895 mini_ranger.cc:68] Stopped Ranger
2026-08-08 02:19:48.541 UTC [8321] LOG: received smart shutdown request
2026-08-08 02:19:48.547 UTC [8321] LOG: background worker "logical replication launcher" (PID 8334) exited with exit code 1
2026-08-08 02:19:48.547 UTC [8327] LOG: shutting down
2026-08-08 02:19:48.778 UTC [8321] LOG: database system is shut down
2026-08-08T02:19:48Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (56698 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:19:48 dist-test-slave-1vmg krb5kdc[9808](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:19:48 dist-test-slave-1vmg krb5kdc[9808](info): set up 2 sockets
Aug 08 02:19:48 dist-test-slave-1vmg krb5kdc[9808](info): commencing operation
krb5kdc: starting...
W20260808 02:19:50.070726 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:19:50 dist-test-slave-1vmg krb5kdc[9808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:19:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:19:50Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.79.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.79.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.79.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/rangeradmin_127.24.79.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.79.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/rangeradmin_127.24.79.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.79.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.79.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.79.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/rangerlookup_127.24.79.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.79.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/rangerlookup_127.24.79.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.79.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.79.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.79.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/HTTP_127.24.79.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.79.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/HTTP_127.24.79.212@KRBTEST.COM.keytab.
I20260808 02:19:50.307684 24895 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task9BOO5x/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/postgres -l logfile start
2026-08-08 02:19:53.861 UTC [9846] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 02:19:53.861 UTC [9846] LOG: listening on IPv4 address "127.24.79.212", port 43207
2026-08-08 02:19:53.870 UTC [9846] LOG: listening on Unix socket "/tmp/.s.PGSQL.43207"
2026-08-08 02:19:53.900 UTC [9850] LOG: database system was shut down at 2026-08-08 02:19:51 UTC
2026-08-08 02:19:53.906 UTC [9846] LOG: database system is ready to accept connections
I20260808 02:19:54.964231 24895 mini_postgres.cc:96] Postgres bound to 43207
2026-08-08 02:19:54.973 UTC [9866] FATAL: database "slave" does not exist
127.24.79.212:43207 - accepting connections
I20260808 02:19:54.974125 24895 mini_ranger.cc:161] Starting Ranger...
I20260808 02:19:54.995287 24895 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 02:19:55.833156 24895 mini_ranger.cc:87] Created ranger Postgres database
I20260808 02:19:55.833238 24895 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ranger-admin
2026-08-08 02:19:57,127 [I] DB FLAVOR :POSTGRES
2026-08-08 02:19:57,128 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 02:19:57,128 [I] Checking connection..
2026-08-08 02:19:57,128 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:43207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 02:19:57,425 [I] Checking connection passed.
2026-08-08 02:19:57,426 [I] --------- Verifying version history table ---------
2026-08-08 02:19:57,426 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:43207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:19:57,736 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 02:19:57,736 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:43207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:19:58,029 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 02:19:58,029 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 02:19:58,029 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:43207/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 02:19:58.345 UTC [9959] WARNING: there is no transaction in progress
2026-08-08 02:19:58,355 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 02:19:58,356 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:43207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:19:58,658 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 02:19:58,659 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 02:19:58,659 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:43207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 02:19:58,980 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:43207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 02:19:59,274 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:43207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1vmg.c.gcp-upstream.internal','N') ;"
2026-08-08 02:19:59,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:43207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 02:19:59,850 [I] Table x_portal_user does not exist in database ranger
2026-08-08 02:19:59,851 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:43207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 02:20:00,136 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 02:20:00,137 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:43207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 02:20:00,432 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:43207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 02:20:00,720 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 02:20:00,720 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:43207/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 02:20:02.000 UTC [10106] WARNING: there is no transaction in progress
2026-08-08 02:20:02.024 UTC [10106] WARNING: there is no transaction in progress
2026-08-08 02:20:02.052 UTC [10106] WARNING: there is no transaction in progress
2026-08-08 02:20:02.074 UTC [10106] WARNING: there is no transaction in progress
2026-08-08 02:20:02.100 UTC [10106] WARNING: there is no transaction in progress
2026-08-08 02:20:02.130 UTC [10106] WARNING: there is no transaction in progress
2026-08-08 02:20:02.214 UTC [10106] WARNING: there is no transaction in progress
2026-08-08 02:20:02.227 UTC [10106] WARNING: there is no transaction in progress
2026-08-08 02:20:02.248 UTC [10106] WARNING: there is no transaction in progress
2026-08-08 02:20:02.264 UTC [10106] WARNING: there is no transaction in progress
2026-08-08 02:20:02.280 UTC [10106] WARNING: there is no transaction in progress
2026-08-08 02:20:02.301 UTC [10106] WARNING: there is no transaction in progress
2026-08-08 02:20:02.315 UTC [10106] WARNING: there is no transaction in progress
2026-08-08 02:20:02.325 UTC [10106] WARNING: there is no transaction in progress
2026-08-08 02:20:02.335 UTC [10106] WARNING: there is no transaction in progress
2026-08-08 02:20:02.346 UTC [10106] WARNING: there is no transaction in progress
2026-08-08 02:20:03,437 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 02:20:03,437 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:43207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 02:20:03,736 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 02:20:03,736 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:43207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 02:20:04,036 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 02:20:04,036 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:43207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 02:20:04,334 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 02:20:04,335 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:43207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 02:20:04,621 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:43207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 02:20:04,923 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:43207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1vmg.c.gcp-upstream.internal';"
2026-08-08 02:20:05,219 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 02:20:05,219 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:43207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 02:20:05,503 [I] DB_PATCHES have already been applied
I20260808 02:20:05.510012 24895 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ranger-admin:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task9BOO5x/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 02:20:05.510104 24895 mini_ranger.cc:193] Using host: 127.24.79.212
I20260808 02:20:05.515646 24895 mini_ranger.cc:227] Ranger admin URL: http://127.24.79.212:38877
Aug 08, 2026 2:20:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:20:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:20:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 2:20:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 2:20:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 2:20:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 2:20:06 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 02:20:06.760598 24895 mini_ranger.cc:160] Time spent starting Ranger: real 11.786s user 0.001s sys 0.014s
Aug 08, 2026 2:20:32 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 2:20:32 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 2:20:32 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 2:20:32 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 2:20:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 2:20:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 2:20:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 2:20:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 2:20:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 2:20:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 2:20:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 2:20:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 2:20:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 2:20:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 2:20:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 2:20:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 2:20:32 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 2:20:33 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 02:20:34.416249 24895 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.79.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.79.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.79.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/rangerkms_127.24.79.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.79.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/rangerkms_127.24.79.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.79.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.79.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.79.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/HTTP_127.24.79.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.79.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/HTTP_127.24.79.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 02:20:34 dist-test-slave-1vmg krb5kdc[9808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155634, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 02:20:34.529641 24895 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 02:20:34.544102 24895 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 02:20:35.722350 24895 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 02:20:35.722431 24895 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ranger-kms
I20260808 02:20:35.722445 24895 mini_ranger_kms.cc:227] Using postgres at 127.24.79.212:43207
2026-08-08 02:20:35,913 [I] DB FLAVOR :POSTGRES
2026-08-08 02:20:35,913 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 02:20:35,914 [I] Checking connection
2026-08-08 02:20:35,914 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:43207/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 02:20:36,208 [I] connection success
2026-08-08 02:20:36,209 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 02:20:36,209 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 02:20:36,209 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:43207/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 02:20:36,501 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 02:20:36,501 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 02:20:36,501 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 02:20:36,501 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.79.212:43207/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 02:20:36,806 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 02:20:37.420230 24895 mini_ranger_kms.cc:310] Created kms service
I20260808 02:20:37.677335 24895 mini_ranger_kms.cc:326] Created kudu user
I20260808 02:20:37.735502 24895 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 02:20:37 dist-test-slave-1vmg krb5kdc[9808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155634, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.79.212@KRBTEST.COM
I20260808 02:20:38.226944 24895 mini_ranger_kms.cc:384] Added ranger policy
I20260808 02:20:38.227128 24895 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ranger-kms:/tmp/dist-test-task9BOO5x/build/debug/bin/postgresql.jar:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task9BOO5x/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 02:20:38.227159 24895 mini_ranger_kms.cc:240] Using host: 127.24.79.212
I20260808 02:20:38.232558 24895 mini_ranger_kms.cc:276] Ranger KMS PID: 10364
I20260808 02:20:38.232664 24895 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.79.212:60923
Aug 08, 2026 2:20:39 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:20:39 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:20:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 2:20:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 2:20:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 2:20:40 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 02:20:40.833258 24895 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.304s user 0.023s sys 0.152s
I20260808 02:20:40.833402 24895 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 02:20:40.833474 24895 mini_ranger_kms.cc:401] 127.24.79.212:60923/kms/v1/keys
Aug 08 02:20:40 dist-test-slave-1vmg krb5kdc[9808](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786155640, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.79.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 02:20:41 dist-test-slave-1vmg krb5kdc[9808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155640, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.79.212@KRBTEST.COM for HTTP/127.24.79.212@KRBTEST.COM
Aug 08 02:20:41 dist-test-slave-1vmg krb5kdc[9808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155640, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.79.212@KRBTEST.COM for HTTP/127.24.79.212@KRBTEST.COM
Aug 08 02:20:44 dist-test-slave-1vmg krb5kdc[9808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:20:44.436230 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:44707
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:41439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:44707
--ranger_config_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.79.212:60923/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:20:44.565531 10434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:20:44.565876 10434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:20:44.565918 10434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:20:44.570336 10434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:20:44.570436 10434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:20:44.570456 10434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:20:44.570472 10434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:20:44.570487 10434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:20:44.576712 10434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:41439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.79.212:60923/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:44707
--ranger_config_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:44707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:20:44.577919 10434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:20:44.579058 10434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:20:44.589394 10445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:44.589416 10441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:44.589421 10442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:44.589663 10434 server_base.cc:1034] running on GCE node
I20260808 02:20:44.590417 10434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:20:44.591770 10434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:20:44.592988 10434 hybrid_clock.cc:648] HybridClock initialized: now 1786155644592965 us; error 41 us; skew 500 ppm
Aug 08 02:20:44 dist-test-slave-1vmg krb5kdc[9808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155644, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:20:44.599898 10434 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:20:44.601161 10434 webserver.cc:460] Webserver started at http://127.24.79.254:42609/ using document root <none> and password file <none>
I20260808 02:20:44.601809 10434 fs_manager.cc:359] Metadata directory not provided
I20260808 02:20:44.601881 10434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:20:44.602115 10434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:20:44 dist-test-slave-1vmg krb5kdc[9808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155644, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for HTTP/127.24.79.212@KRBTEST.COM
I20260808 02:20:44.669019 10434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "b187e43a949f4677a8f14737b32cabb9"
format_stamp: "Formatted at 2026-08-08 02:20:44 on dist-test-slave-1vmg"
server_key: "8a2db82dec1944519954420fd56f385a"
server_key_iv: "bb184b53570ffb2b4c990c1a47c6d2fe"
server_key_version: "kuduclusterkey@0"
I20260808 02:20:44.669996 10434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "b187e43a949f4677a8f14737b32cabb9"
format_stamp: "Formatted at 2026-08-08 02:20:44 on dist-test-slave-1vmg"
server_key: "8a2db82dec1944519954420fd56f385a"
server_key_iv: "bb184b53570ffb2b4c990c1a47c6d2fe"
server_key_version: "kuduclusterkey@0"
I20260808 02:20:44.676489 10434 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260808 02:20:44.698062 10458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:44.700078 10434 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260808 02:20:44.700291 10434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "b187e43a949f4677a8f14737b32cabb9"
format_stamp: "Formatted at 2026-08-08 02:20:44 on dist-test-slave-1vmg"
server_key: "8a2db82dec1944519954420fd56f385a"
server_key_iv: "bb184b53570ffb2b4c990c1a47c6d2fe"
server_key_version: "kuduclusterkey@0"
I20260808 02:20:44.700479 10434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:20:44.730707 10434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:20:44.734578 10434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:20:44.734823 10434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:20:44.743616 10434 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:44707
I20260808 02:20:44.743651 10546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:44707 every 8 connection(s)
I20260808 02:20:44.744887 10434 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:20:44.747709 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 10434
I20260808 02:20:44.747850 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:20:44.748313 24895 external_mini_cluster.cc:1428] Setting key a0079207c6336e7bb37e6825ff451270
I20260808 02:20:44.749284 10547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 02:20:44 dist-test-slave-1vmg krb5kdc[9808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:20:44.758709 10547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9: Bootstrap starting.
I20260808 02:20:44.762353 10547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9: Neither blocks nor log segments found. Creating new log.
I20260808 02:20:44.764014 10550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:44.750067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33386 (local address 127.24.79.254:44707)
0808 02:20:44.750576 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:44.750595 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:44.750636 (+ 41us) server_negotiation.cc:407] Connection header received
0808 02:20:44.751410 (+ 774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:44.751434 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:44.751857 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:44.752234 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:44.753169 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:44.754300 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:44.755298 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:44.755867 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:44.756035 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:44.758894 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:44.758902 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:44.758956 (+ 54us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:44.759002 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:44.760703 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:44.761243 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:44.761249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:44.761255 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:44.761387 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:44.761690 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:44.761693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:44.761694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:44.762197 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:44.762408 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:44.762750 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 02:20:44.763154 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":168,"threads_started":1}
I20260808 02:20:44.764115 10547 log.cc:824] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9: Log is configured to *not* fsync() on all Append() calls
I20260808 02:20:44.767246 10547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9: No bootstrap required, opened a new log
I20260808 02:20:44.771994 10547 raft_consensus.cc:348] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e43a949f4677a8f14737b32cabb9" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 44707 } }
I20260808 02:20:44.772428 10547 raft_consensus.cc:374] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:20:44.772552 10547 raft_consensus.cc:729] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b187e43a949f4677a8f14737b32cabb9, State: Initialized, Role: FOLLOWER
I20260808 02:20:44.773234 10547 consensus_queue.cc:260] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e43a949f4677a8f14737b32cabb9" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 44707 } }
I20260808 02:20:44.773460 10547 raft_consensus.cc:388] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:20:44.773550 10547 raft_consensus.cc:482] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:20:44.773677 10547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:20:44.774885 10547 raft_consensus.cc:504] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e43a949f4677a8f14737b32cabb9" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 44707 } }
I20260808 02:20:44.775358 10547 leader_election.cc:302] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b187e43a949f4677a8f14737b32cabb9; no voters:
I20260808 02:20:44.775749 10547 leader_election.cc:288] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:20:44.775892 10553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:20:44.776165 10553 raft_consensus.cc:686] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [term 1 LEADER]: Becoming Leader. State: Replica: b187e43a949f4677a8f14737b32cabb9, State: Running, Role: LEADER
I20260808 02:20:44.776575 10553 consensus_queue.cc:237] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e43a949f4677a8f14737b32cabb9" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 44707 } }
I20260808 02:20:44.777182 10547 sys_catalog.cc:564] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:20:44.778421 10555 sys_catalog.cc:455] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b187e43a949f4677a8f14737b32cabb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e43a949f4677a8f14737b32cabb9" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 44707 } } }
I20260808 02:20:44.778421 10556 sys_catalog.cc:455] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b187e43a949f4677a8f14737b32cabb9. Latest consensus state: current_term: 1 leader_uuid: "b187e43a949f4677a8f14737b32cabb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e43a949f4677a8f14737b32cabb9" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 44707 } } }
I20260808 02:20:44.778693 10556 sys_catalog.cc:458] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [sys.catalog]: This master's current role is: LEADER
I20260808 02:20:44.778693 10555 sys_catalog.cc:458] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [sys.catalog]: This master's current role is: LEADER
I20260808 02:20:44.779260 10569 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:20:44.779644 10547 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 02:20:44.786286 10569 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:20:44.795338 10569 catalog_manager.cc:1347] Generated new cluster ID: 5c058f4d0fac40f891bfb3df46ffe639
I20260808 02:20:44.795430 10569 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:20:44.812049 10569 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:20:44.813584 10569 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:20:44.827116 10569 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9: Generated new TSK 0
I20260808 02:20:44.828315 10569 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.79.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
I20260808 02:20:45.600114 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.79.212:60923/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:44707
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:41439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:20:45.742748 10609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:20:45.743127 10609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:20:45.743244 10609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:20:45.749200 10609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:20:45.749316 10609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:20:45.749410 10609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:20:45.759169 10609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:41439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.79.212:60923/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:44707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:20:45.761159 10609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:20:45.762961 10609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:20:45.776628 10615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:45.776670 10614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:45.776847 10618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:45.779094 10609 server_base.cc:1034] running on GCE node
I20260808 02:20:45.780007 10609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:20:45.781003 10609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:20:45.782488 10609 hybrid_clock.cc:648] HybridClock initialized: now 1786155645782451 us; error 60 us; skew 500 ppm
Aug 08 02:20:45 dist-test-slave-1vmg krb5kdc[9808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155645, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:20:45.786108 10609 init.cc:377] Logged in from keytab as kudu/127.24.79.193@KRBTEST.COM (short username kudu)
I20260808 02:20:45.787338 10609 webserver.cc:460] Webserver started at http://127.24.79.193:38507/ using document root <none> and password file <none>
I20260808 02:20:45.787985 10609 fs_manager.cc:359] Metadata directory not provided
I20260808 02:20:45.788055 10609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:20:45.788275 10609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:20:45 dist-test-slave-1vmg krb5kdc[9808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155645, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for HTTP/127.24.79.212@KRBTEST.COM
I20260808 02:20:45.806483 10609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "e94c5ae1175943099a58d0b426d0472a"
format_stamp: "Formatted at 2026-08-08 02:20:45 on dist-test-slave-1vmg"
server_key: "d073bb690f5fd6ae5c205c73b770e27b"
server_key_iv: "93c05657744f9eb94894c29845f65353"
server_key_version: "kuduclusterkey@0"
I20260808 02:20:45.807660 10609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "e94c5ae1175943099a58d0b426d0472a"
format_stamp: "Formatted at 2026-08-08 02:20:45 on dist-test-slave-1vmg"
server_key: "d073bb690f5fd6ae5c205c73b770e27b"
server_key_iv: "93c05657744f9eb94894c29845f65353"
server_key_version: "kuduclusterkey@0"
I20260808 02:20:45.816862 10609 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.005s sys 0.000s
I20260808 02:20:45.839854 10629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:45.841919 10609 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:20:45.842218 10609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "e94c5ae1175943099a58d0b426d0472a"
format_stamp: "Formatted at 2026-08-08 02:20:45 on dist-test-slave-1vmg"
server_key: "d073bb690f5fd6ae5c205c73b770e27b"
server_key_iv: "93c05657744f9eb94894c29845f65353"
server_key_version: "kuduclusterkey@0"
I20260808 02:20:45.842486 10609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:20:45.880268 10609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:20:45.884738 10609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:20:45.885063 10609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:20:45.886004 10609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:20:45.887404 10609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:20:45.887543 10609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260808 02:20:45.887679 10609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:20:45.887771 10609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:45.900388 10609 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:45865
I20260808 02:20:45.901971 10609 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:20:45.902782 10792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:45865 every 8 connection(s)
I20260808 02:20:45.911145 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 10609
I20260808 02:20:45.911288 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:20:45.911597 24895 external_mini_cluster.cc:1428] Setting key fa5991432575fc84760a76599d5ac851
Aug 08 02:20:45 dist-test-slave-1vmg krb5kdc[9808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155645, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:20:45.930583 10799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:45.907465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:39127 (local address 127.24.79.254:44707)
0808 02:20:45.910306 (+ 2841us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:45.910312 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:45.910329 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:20:45.910382 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:45.910385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:45.910443 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:45.910542 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:45.916917 (+ 6375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:45.917870 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:45.920096 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:45.920483 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:45.920589 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:45.925145 (+ 4556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:45.925150 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:45.925153 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:45.925191 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:45.927250 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:45.928170 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:45.928174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:45.928176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:45.928312 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:45.928876 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:45.928880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:45.928882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:45.929168 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:45.929279 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:45.930185 (+ 906us) server_negotiation.cc:299] Negotiation successful
0808 02:20:45.930376 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2736,"thread_start_us":111,"threads_started":1}
WARNING: no policy specified for kudu/127.24.79.194@KRBTEST.COM; defaulting to no policy
I20260808 02:20:45.932905 10797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:45.906045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:44707 (local address 127.24.79.193:39127)
0808 02:20:45.908416 (+ 2371us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:45.908486 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:45.909739 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:45.912370 (+ 2631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:45.912384 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:45.912814 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:20:45.916629 (+ 3815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:45.916664 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:45.918127 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:45.918131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:45.919877 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:45.919897 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:45.920892 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:45.920897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:45.921148 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:45.921995 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:20:45.922021 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:20:45.924846 (+ 2825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:20:45.927484 (+ 2638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:45.927494 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:45.927521 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:45.928013 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:45.928455 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:45.928458 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:45.928461 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:45.928730 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:45.929371 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:20:45.929381 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:20:45.929828 (+ 447us) client_negotiation.cc:769] Sending connection context
0808 02:20:45.931099 (+ 1271us) client_negotiation.cc:240] Negotiation successful
0808 02:20:45.931575 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2095,"thread_start_us":182,"threads_started":1}
I20260808 02:20:45.934638 10793 heartbeater.cc:347] Connected to a master server at 127.24.79.254:44707
I20260808 02:20:45.934980 10793 heartbeater.cc:464] Registering TS with master...
I20260808 02:20:45.935807 10793 heartbeater.cc:511] Master 127.24.79.254:44707 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.194@KRBTEST.COM" created.
I20260808 02:20:45.938205 10480 ts_manager.cc:194] Registered new tserver with Master: e94c5ae1175943099a58d0b426d0472a (127.24.79.193:45865)
I20260808 02:20:45.939842 10480 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.193@KRBTEST.COM'} at 127.24.79.193:39127
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
I20260808 02:20:45.959589 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.79.212:60923/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:44707
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:41439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:20:45 dist-test-slave-1vmg krb5kdc[9808](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786155645, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 02:20:46.125527 10802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:20:46.125988 10802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:20:46.126062 10802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:20:46.133651 10802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:20:46.133790 10802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:20:46.133888 10802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:20:46.144348 10802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:41439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.79.212:60923/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:44707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:20:46.146147 10802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:20:46.147658 10802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:20:46.161230 10811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:46.161230 10810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:46.161993 10817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:46.162365 10802 server_base.cc:1034] running on GCE node
I20260808 02:20:46.163015 10802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:20:46.163852 10802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:20:46.165179 10802 hybrid_clock.cc:648] HybridClock initialized: now 1786155646165118 us; error 82 us; skew 500 ppm
Aug 08 02:20:46 dist-test-slave-1vmg krb5kdc[9808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155646, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:20:46.168737 10802 init.cc:377] Logged in from keytab as kudu/127.24.79.194@KRBTEST.COM (short username kudu)
I20260808 02:20:46.170308 10802 webserver.cc:460] Webserver started at http://127.24.79.194:37513/ using document root <none> and password file <none>
I20260808 02:20:46.171383 10802 fs_manager.cc:359] Metadata directory not provided
I20260808 02:20:46.171687 10802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:20:46.172091 10802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:20:46 dist-test-slave-1vmg krb5kdc[9808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155646, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for HTTP/127.24.79.212@KRBTEST.COM
I20260808 02:20:46.192899 10802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "5eb44967e0264963918977ccd1f2b9be"
format_stamp: "Formatted at 2026-08-08 02:20:46 on dist-test-slave-1vmg"
server_key: "1e8db89b7fd5758242939b99da8d8cd5"
server_key_iv: "41d09692d55a64d0a926701f14cffc0d"
server_key_version: "kuduclusterkey@0"
I20260808 02:20:46.193856 10802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "5eb44967e0264963918977ccd1f2b9be"
format_stamp: "Formatted at 2026-08-08 02:20:46 on dist-test-slave-1vmg"
server_key: "1e8db89b7fd5758242939b99da8d8cd5"
server_key_iv: "41d09692d55a64d0a926701f14cffc0d"
server_key_version: "kuduclusterkey@0"
I20260808 02:20:46.201125 10802 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260808 02:20:46.237828 10826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:46.240108 10802 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260808 02:20:46.240310 10802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "5eb44967e0264963918977ccd1f2b9be"
format_stamp: "Formatted at 2026-08-08 02:20:46 on dist-test-slave-1vmg"
server_key: "1e8db89b7fd5758242939b99da8d8cd5"
server_key_iv: "41d09692d55a64d0a926701f14cffc0d"
server_key_version: "kuduclusterkey@0"
I20260808 02:20:46.240487 10802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:20:46.274125 10802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:20:46.279022 10802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:20:46.279281 10802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:20:46.280189 10802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:20:46.281456 10802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:20:46.281533 10802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:46.281602 10802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:20:46.281639 10802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:46.293741 10802 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:37583
I20260808 02:20:46.293795 10983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:37583 every 8 connection(s)
I20260808 02:20:46.295675 10802 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:20:46.297247 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 10802
I20260808 02:20:46.297382 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:20:46.297729 24895 external_mini_cluster.cc:1428] Setting key 34a792b155ff5fa868b9b1b3f0a7a6ff
Aug 08 02:20:46 dist-test-slave-1vmg krb5kdc[9808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155646, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.79.195@KRBTEST.COM; defaulting to no policy
I20260808 02:20:46.316627 10799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:46.302349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:35537 (local address 127.24.79.254:44707)
0808 02:20:46.302510 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:46.302515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:46.302530 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:20:46.302593 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:46.302596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:46.302647 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:46.302727 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:46.304454 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:46.305359 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:46.307391 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:46.307783 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:46.307916 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:46.311574 (+ 3658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:46.311578 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:46.311581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:46.311621 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:46.313600 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:46.314438 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:46.314443 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:46.314444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:46.314557 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:46.315117 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:46.315121 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:46.315123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:46.315355 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:46.315452 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:46.316254 (+ 802us) server_negotiation.cc:299] Negotiation successful
0808 02:20:46.316420 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.195@KRBTEST.COM" created.
I20260808 02:20:46.317906 10987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:46.299360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:44707 (local address 127.24.79.194:35537)
0808 02:20:46.301124 (+ 1764us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:46.301202 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:46.302294 (+ 1092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:46.303011 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:46.303022 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:46.303451 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:20:46.304230 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:46.304251 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:46.305561 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:46.305566 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:46.307162 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:46.307176 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:46.308177 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:46.308182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:46.308411 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:46.309219 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:20:46.309244 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:20:46.311269 (+ 2025us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:20:46.313810 (+ 2541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:46.313819 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:46.313847 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:46.314291 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:46.314706 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:46.314709 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:46.314712 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:46.315002 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:46.315484 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:20:46.315494 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:20:46.315897 (+ 403us) client_negotiation.cc:769] Sending connection context
0808 02:20:46.316149 (+ 252us) client_negotiation.cc:240] Negotiation successful
0808 02:20:46.316544 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1428,"thread_start_us":209,"threads_started":1}
I20260808 02:20:46.319576 10985 heartbeater.cc:347] Connected to a master server at 127.24.79.254:44707
I20260808 02:20:46.319958 10985 heartbeater.cc:464] Registering TS with master...
I20260808 02:20:46.320889 10985 heartbeater.cc:511] Master 127.24.79.254:44707 requested a full tablet report, sending...
I20260808 02:20:46.322599 10480 ts_manager.cc:194] Registered new tserver with Master: 5eb44967e0264963918977ccd1f2b9be (127.24.79.194:37583)
I20260808 02:20:46.323576 10480 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.194@KRBTEST.COM'} at 127.24.79.194:35537
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
I20260808 02:20:46.351780 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.79.212:60923/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:44707
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:41439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:20:46 dist-test-slave-1vmg krb5kdc[9808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155645, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for HTTP/127.24.79.212@KRBTEST.COM
W20260808 02:20:46.556286 10991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:20:46.556814 10991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:20:46.556952 10991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:20:46.564256 10991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:20:46.564549 10991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:20:46.564693 10991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:20:46.574446 10991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:41439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.79.212:60923/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:44707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:20:46.576294 10991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:20:46.578290 10991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:20:46.609477 10999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:46.609477 11001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:46.610450 10998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:46.613328 10991 server_base.cc:1034] running on GCE node
I20260808 02:20:46.614022 10991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:20:46.615015 10991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:20:46.630632 10991 hybrid_clock.cc:648] HybridClock initialized: now 1786155646630510 us; error 133 us; skew 500 ppm
Aug 08 02:20:46 dist-test-slave-1vmg krb5kdc[9808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155646, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:20:46.635552 10991 init.cc:377] Logged in from keytab as kudu/127.24.79.195@KRBTEST.COM (short username kudu)
I20260808 02:20:46.637096 10991 webserver.cc:460] Webserver started at http://127.24.79.195:39037/ using document root <none> and password file <none>
I20260808 02:20:46.638000 10991 fs_manager.cc:359] Metadata directory not provided
I20260808 02:20:46.638132 10991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:20:46.638433 10991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:20:46 dist-test-slave-1vmg krb5kdc[9808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155646, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for HTTP/127.24.79.212@KRBTEST.COM
I20260808 02:20:46.662814 10991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "cdbd424f32904948932ab70a1648a407"
format_stamp: "Formatted at 2026-08-08 02:20:46 on dist-test-slave-1vmg"
server_key: "b2683c7976ce60ab373124b488b8ce8d"
server_key_iv: "39b2253df6b8c89d410853d509af7828"
server_key_version: "kuduclusterkey@0"
I20260808 02:20:46.664125 10991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "cdbd424f32904948932ab70a1648a407"
format_stamp: "Formatted at 2026-08-08 02:20:46 on dist-test-slave-1vmg"
server_key: "b2683c7976ce60ab373124b488b8ce8d"
server_key_iv: "39b2253df6b8c89d410853d509af7828"
server_key_version: "kuduclusterkey@0"
I20260808 02:20:46.680830 10991 fs_manager.cc:691] Time spent creating directory manager: real 0.016s user 0.005s sys 0.001s
I20260808 02:20:46.707271 11009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:46.709533 10991 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.002s sys 0.000s
I20260808 02:20:46.709759 10991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "cdbd424f32904948932ab70a1648a407"
format_stamp: "Formatted at 2026-08-08 02:20:46 on dist-test-slave-1vmg"
server_key: "b2683c7976ce60ab373124b488b8ce8d"
server_key_iv: "39b2253df6b8c89d410853d509af7828"
server_key_version: "kuduclusterkey@0"
I20260808 02:20:46.709957 10991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:20:46.760686 10991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:20:46.778653 10991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:20:46.778940 10991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:20:46.779814 10991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:20:46.781096 10991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:20:46.781167 10991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:46.781232 10991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:20:46.781263 10991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:46.804773 10991 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:42173
I20260808 02:20:46.806210 10991 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:20:46.807608 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 10991
I20260808 02:20:46.807744 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:20:46.808055 24895 external_mini_cluster.cc:1428] Setting key 984216535ce44a811d1b0e9ea292e4a7
I20260808 02:20:46.813702 11172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:42173 every 8 connection(s)
Aug 08 02:20:46 dist-test-slave-1vmg krb5kdc[9808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155646, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:20:46.854169 11176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:46.816560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:44707 (local address 127.24.79.195:55475)
0808 02:20:46.823618 (+ 7058us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:46.823719 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:46.825010 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:46.825992 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:46.826004 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:46.826470 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:20:46.827360 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:46.827389 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:46.840454 (+ 13065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:46.840459 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:46.842301 (+ 1842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:46.842328 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:46.843477 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:46.843484 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:46.843781 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:46.844683 (+ 902us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:20:46.844713 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:20:46.846772 (+ 2059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:20:46.849363 (+ 2591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:46.849373 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:46.849412 (+ 39us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:46.849943 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:46.850455 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:46.850458 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:46.850460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:46.850721 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:46.851410 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:20:46.851419 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:20:46.851882 (+ 463us) client_negotiation.cc:769] Sending connection context
0808 02:20:46.852161 (+ 279us) client_negotiation.cc:240] Negotiation successful
0808 02:20:46.852663 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6701,"thread_start_us":160,"threads_started":1}
I20260808 02:20:46.855176 10799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:46.815914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:55475 (local address 127.24.79.254:44707)
0808 02:20:46.816071 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:46.816075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:46.823850 (+ 7775us) server_negotiation.cc:407] Connection header received
0808 02:20:46.825336 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:46.825340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:46.825404 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:46.825459 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:46.839151 (+ 13692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:46.840143 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:46.842587 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:46.843027 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:46.843137 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:46.847116 (+ 3979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:46.847121 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:46.847124 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:46.847162 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:46.849170 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:46.850189 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:46.850194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:46.850196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:46.850320 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:46.850883 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:46.850887 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:46.850889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:46.851225 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:46.851333 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:46.854770 (+ 3437us) server_negotiation.cc:299] Negotiation successful
0808 02:20:46.855001 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 02:20:46.857533 11173 heartbeater.cc:347] Connected to a master server at 127.24.79.254:44707
I20260808 02:20:46.857906 11173 heartbeater.cc:464] Registering TS with master...
I20260808 02:20:46.858865 11173 heartbeater.cc:511] Master 127.24.79.254:44707 requested a full tablet report, sending...
I20260808 02:20:46.860569 10480 ts_manager.cc:194] Registered new tserver with Master: cdbd424f32904948932ab70a1648a407 (127.24.79.195:42173)
I20260808 02:20:46.861088 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:20:46.861482 10480 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.195@KRBTEST.COM'} at 127.24.79.195:55475
I20260808 02:20:46.861945 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 10609
I20260808 02:20:46.886335 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 10802
I20260808 02:20:46.902010 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 10991
I20260808 02:20:46.914233 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 10434
I20260808 02:20:46.935216 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:44707
--webserver_interface=127.24.79.254
--webserver_port=42609
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:41439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:44707
--ranger_config_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.79.212:60923/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.79.212:60923/kms with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:20:47.078557 11177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:20:47.078964 11177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:20:47.079036 11177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:20:47.083909 11177 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:20:47.084024 11177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:20:47.084055 11177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:20:47.084077 11177 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:20:47.084097 11177 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:20:47.090444 11177 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:41439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.79.212:60923/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:44707
--ranger_config_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:44707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=42609
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:20:47.091854 11177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:20:47.093482 11177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:20:47.105929 11185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:47.106001 11186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:47.105929 11188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:47.106554 11177 server_base.cc:1034] running on GCE node
I20260808 02:20:47.107378 11177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:20:47.108786 11177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:20:47.110026 11177 hybrid_clock.cc:648] HybridClock initialized: now 1786155647110001 us; error 44 us; skew 500 ppm
Aug 08 02:20:47 dist-test-slave-1vmg krb5kdc[9808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155647, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:20:47.113452 11177 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:20:47.114651 11177 webserver.cc:460] Webserver started at http://127.24.79.254:42609/ using document root <none> and password file <none>
I20260808 02:20:47.115407 11177 fs_manager.cc:359] Metadata directory not provided
I20260808 02:20:47.115500 11177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 02:20:47 dist-test-slave-1vmg krb5kdc[9808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155647, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for HTTP/127.24.79.212@KRBTEST.COM
I20260808 02:20:47.164309 11177 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:20:47.167182 11202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:47.168740 11177 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:20:47.168916 11177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "b187e43a949f4677a8f14737b32cabb9"
format_stamp: "Formatted at 2026-08-08 02:20:44 on dist-test-slave-1vmg"
server_key: "8a2db82dec1944519954420fd56f385a"
server_key_iv: "bb184b53570ffb2b4c990c1a47c6d2fe"
server_key_version: "kuduclusterkey@0"
I20260808 02:20:47.169579 11177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:20:47.181195 11177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:20:47.184867 11177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:20:47.185096 11177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:20:47.194092 11177 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:44707
I20260808 02:20:47.194094 11281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:44707 every 8 connection(s)
I20260808 02:20:47.195864 11177 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:20:47.196547 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 11177
I20260808 02:20:47.197032 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.79.212:60923/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:45865
--local_ip_for_outbound_sockets=127.24.79.193
--tserver_master_addrs=127.24.79.254:44707
--webserver_port=38507
--webserver_interface=127.24.79.193
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:41439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.79.212:60923/kms with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:20:47.202730 11282 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:20:47.205945 11282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9: Bootstrap starting.
I20260808 02:20:47.230386 11282 log.cc:824] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9: Log is configured to *not* fsync() on all Append() calls
I20260808 02:20:47.241149 11282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:20:47.241835 11282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9: Bootstrap complete.
I20260808 02:20:47.248075 11282 raft_consensus.cc:348] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e43a949f4677a8f14737b32cabb9" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 44707 } }
I20260808 02:20:47.248735 11282 raft_consensus.cc:729] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b187e43a949f4677a8f14737b32cabb9, State: Initialized, Role: FOLLOWER
I20260808 02:20:47.249431 11282 consensus_queue.cc:260] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e43a949f4677a8f14737b32cabb9" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 44707 } }
I20260808 02:20:47.249615 11282 raft_consensus.cc:388] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:20:47.249694 11282 raft_consensus.cc:482] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:20:47.249809 11282 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:20:47.253211 11282 raft_consensus.cc:504] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e43a949f4677a8f14737b32cabb9" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 44707 } }
I20260808 02:20:47.253706 11282 leader_election.cc:302] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b187e43a949f4677a8f14737b32cabb9; no voters:
I20260808 02:20:47.254150 11282 leader_election.cc:288] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 02:20:47.254418 11287 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 02:20:47.254956 11287 raft_consensus.cc:686] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [term 2 LEADER]: Becoming Leader. State: Replica: b187e43a949f4677a8f14737b32cabb9, State: Running, Role: LEADER
I20260808 02:20:47.255436 11287 consensus_queue.cc:237] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e43a949f4677a8f14737b32cabb9" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 44707 } }
I20260808 02:20:47.255817 11282 sys_catalog.cc:564] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:20:47.258780 11288 sys_catalog.cc:455] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b187e43a949f4677a8f14737b32cabb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e43a949f4677a8f14737b32cabb9" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 44707 } } }
I20260808 02:20:47.259039 11288 sys_catalog.cc:458] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [sys.catalog]: This master's current role is: LEADER
I20260808 02:20:47.258862 11282 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 02:20:47.260578 11289 sys_catalog.cc:455] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b187e43a949f4677a8f14737b32cabb9. Latest consensus state: current_term: 2 leader_uuid: "b187e43a949f4677a8f14737b32cabb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187e43a949f4677a8f14737b32cabb9" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 44707 } } }
I20260808 02:20:47.260833 11289 sys_catalog.cc:458] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9 [sys.catalog]: This master's current role is: LEADER
I20260808 02:20:47.261268 11304 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:20:47.268323 11304 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:20:47.269682 11304 catalog_manager.cc:1259] Loaded cluster ID: 5c058f4d0fac40f891bfb3df46ffe639
I20260808 02:20:47.269881 11304 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:20:47.273003 11304 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:20:47.274938 11304 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9: Loaded TSK: 0
I20260808 02:20:47.276746 11304 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 02:20:47.362424 11284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:20:47.362900 11284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:20:47.363027 11284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:20:47.369817 11284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:20:47.369937 11284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:20:47.370030 11284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:20:47.378997 11284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:41439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.79.212:60923/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:45865
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=38507
--tserver_master_addrs=127.24.79.254:44707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:20:47.380676 11284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:20:47.382174 11284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:20:47.395005 11334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:47.395080 11330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:47.395078 11329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:47.395493 11284 server_base.cc:1034] running on GCE node
I20260808 02:20:47.396063 11284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:20:47.396896 11284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:20:47.398129 11284 hybrid_clock.cc:648] HybridClock initialized: now 1786155647398093 us; error 157 us; skew 500 ppm
Aug 08 02:20:47 dist-test-slave-1vmg krb5kdc[9808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155647, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:20:47.401938 11284 init.cc:377] Logged in from keytab as kudu/127.24.79.193@KRBTEST.COM (short username kudu)
I20260808 02:20:47.403367 11284 webserver.cc:460] Webserver started at http://127.24.79.193:38507/ using document root <none> and password file <none>
I20260808 02:20:47.404217 11284 fs_manager.cc:359] Metadata directory not provided
I20260808 02:20:47.404332 11284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 02:20:47 dist-test-slave-1vmg krb5kdc[9808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155647, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for HTTP/127.24.79.212@KRBTEST.COM
I20260808 02:20:47.436832 11284 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.002s sys 0.001s
I20260808 02:20:47.441318 11345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:47.443341 11284 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260808 02:20:47.443528 11284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "e94c5ae1175943099a58d0b426d0472a"
format_stamp: "Formatted at 2026-08-08 02:20:45 on dist-test-slave-1vmg"
server_key: "d073bb690f5fd6ae5c205c73b770e27b"
server_key_iv: "93c05657744f9eb94894c29845f65353"
server_key_version: "kuduclusterkey@0"
I20260808 02:20:47.444427 11284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:20:47.465807 11284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:20:47.470160 11284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:20:47.470422 11284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:20:47.471741 11284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:20:47.473094 11284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:20:47.473178 11284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:47.473245 11284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:20:47.473274 11284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:47.488505 11284 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:45865
I20260808 02:20:47.488653 11496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:45865 every 8 connection(s)
I20260808 02:20:47.490423 11284 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:20:47.491393 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 11284
I20260808 02:20:47.491875 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.79.212:60923/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:37583
--local_ip_for_outbound_sockets=127.24.79.194
--tserver_master_addrs=127.24.79.254:44707
--webserver_port=37513
--webserver_interface=127.24.79.194
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:41439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.79.212:60923/kms with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:20:47 dist-test-slave-1vmg krb5kdc[9808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155647, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:20:47.537037 11505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:47.505950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:44707 (local address 127.24.79.193:36515)
0808 02:20:47.515845 (+ 9895us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:47.515952 (+ 107us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:47.517214 (+ 1262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:47.519913 (+ 2699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:47.519924 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:47.520339 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:20:47.521069 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:47.521092 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:47.523119 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:47.523124 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:47.524495 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:47.524513 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:47.526636 (+ 2123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:47.526641 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:47.526900 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:47.527756 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:20:47.527782 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:20:47.529832 (+ 2050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:20:47.532670 (+ 2838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:47.532679 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:47.532695 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:47.533118 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:47.533630 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:47.533634 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:47.533636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:47.533852 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:47.534798 (+ 946us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:20:47.534805 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:20:47.535182 (+ 377us) client_negotiation.cc:769] Sending connection context
0808 02:20:47.535479 (+ 297us) client_negotiation.cc:240] Negotiation successful
0808 02:20:47.535896 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9555,"thread_start_us":243,"threads_started":1}
I20260808 02:20:47.539426 11506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:47.507991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:36515 (local address 127.24.79.254:44707)
0808 02:20:47.515361 (+ 7370us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:47.515394 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:47.517702 (+ 2308us) server_negotiation.cc:407] Connection header received
0808 02:20:47.518221 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:47.518250 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:47.518787 (+ 537us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:47.519280 (+ 493us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:47.521336 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:47.522876 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:47.524720 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:47.526040 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:47.526318 (+ 278us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:47.530166 (+ 3848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:47.530174 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:47.530194 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:47.530240 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:47.532462 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:47.533299 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:47.533306 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:47.533313 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:47.533497 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:47.533977 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:47.533981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:47.533982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:47.534562 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:47.534716 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:47.537873 (+ 3157us) server_negotiation.cc:299] Negotiation successful
0808 02:20:47.538258 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7089,"thread_start_us":146,"threads_started":1}
I20260808 02:20:47.542642 11497 heartbeater.cc:347] Connected to a master server at 127.24.79.254:44707
I20260808 02:20:47.542984 11497 heartbeater.cc:464] Registering TS with master...
I20260808 02:20:47.543704 11497 heartbeater.cc:511] Master 127.24.79.254:44707 requested a full tablet report, sending...
W20260808 02:20:47.546983 11497 heartbeater.cc:644] Failed to heartbeat to 127.24.79.254:44707 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 02:20:47.547174 11497 heartbeater.cc:464] Registering TS with master...
I20260808 02:20:47.547432 11497 heartbeater.cc:511] Master 127.24.79.254:44707 requested a full tablet report, sending...
I20260808 02:20:47.548751 11497 heartbeater.cc:464] Registering TS with master...
I20260808 02:20:47.549026 11497 heartbeater.cc:511] Master 127.24.79.254:44707 requested a full tablet report, sending...
W20260808 02:20:47.550248 11497 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:20:47.657526 11499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:20:47.657848 11499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:20:47.657912 11499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:20:47.662420 11499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:20:47.662501 11499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:20:47.662580 11499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:20:47.668829 11499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:41439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.79.212:60923/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:37583
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=37513
--tserver_master_addrs=127.24.79.254:44707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:20:47.670421 11499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:20:47.671979 11499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:20:47.684897 11513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:47.684883 11517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:47.686393 11514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:47.687294 11499 server_base.cc:1034] running on GCE node
I20260808 02:20:47.687950 11499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:20:47.688719 11499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:20:47.689929 11499 hybrid_clock.cc:648] HybridClock initialized: now 1786155647689877 us; error 73 us; skew 500 ppm
Aug 08 02:20:47 dist-test-slave-1vmg krb5kdc[9808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155647, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:20:47.693536 11499 init.cc:377] Logged in from keytab as kudu/127.24.79.194@KRBTEST.COM (short username kudu)
I20260808 02:20:47.695014 11499 webserver.cc:460] Webserver started at http://127.24.79.194:37513/ using document root <none> and password file <none>
I20260808 02:20:47.695899 11499 fs_manager.cc:359] Metadata directory not provided
I20260808 02:20:47.696033 11499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 02:20:47 dist-test-slave-1vmg krb5kdc[9808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155647, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for HTTP/127.24.79.212@KRBTEST.COM
I20260808 02:20:47.718216 11499 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:20:47.723390 11526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:47.725374 11499 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260808 02:20:47.725545 11499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "5eb44967e0264963918977ccd1f2b9be"
format_stamp: "Formatted at 2026-08-08 02:20:46 on dist-test-slave-1vmg"
server_key: "1e8db89b7fd5758242939b99da8d8cd5"
server_key_iv: "41d09692d55a64d0a926701f14cffc0d"
server_key_version: "kuduclusterkey@0"
I20260808 02:20:47.726390 11499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:20:47.753682 11499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:20:47.757493 11499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:20:47.757723 11499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:20:47.758566 11499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:20:47.759778 11499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:20:47.759848 11499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:47.759912 11499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:20:47.759944 11499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:47.773602 11499 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:37583
I20260808 02:20:47.775220 11499 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:20:47.775133 11681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:37583 every 8 connection(s)
I20260808 02:20:47.782758 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 11499
I20260808 02:20:47.783335 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.79.212:60923/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:42173
--local_ip_for_outbound_sockets=127.24.79.195
--tserver_master_addrs=127.24.79.254:44707
--webserver_port=39037
--webserver_interface=127.24.79.195
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:41439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.79.212:60923/kms with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:20:47 dist-test-slave-1vmg krb5kdc[9808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155647, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:20:47.793668 11506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:47.777506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:38357 (local address 127.24.79.254:44707)
0808 02:20:47.777665 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:47.777670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:47.779005 (+ 1335us) server_negotiation.cc:407] Connection header received
0808 02:20:47.780515 (+ 1510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:47.780520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:47.780592 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:47.780653 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:47.782703 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:47.783622 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:47.785637 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:47.786034 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:47.786143 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:47.789544 (+ 3401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:47.789554 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:47.789556 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:47.789590 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:47.791211 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:47.791868 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:47.791872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:47.791874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:47.791965 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:47.792342 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:47.792346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:47.792348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:47.792615 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:47.792712 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:47.793326 (+ 614us) server_negotiation.cc:299] Negotiation successful
0808 02:20:47.793476 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:20:47.794602 11684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:47.778118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:44707 (local address 127.24.79.194:38357)
0808 02:20:47.778752 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:47.778820 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:47.780208 (+ 1388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:47.781187 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:47.781197 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:47.781616 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:20:47.782472 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:47.782498 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:47.783824 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:47.783827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:47.785447 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:47.785461 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:47.786410 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:47.786414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:47.786680 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:47.787528 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:20:47.787554 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:20:47.789309 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:20:47.791352 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:47.791361 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:47.791383 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:47.791769 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:47.792065 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:47.792069 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:47.792071 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:47.792240 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:47.792730 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:20:47.792738 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:20:47.793061 (+ 323us) client_negotiation.cc:769] Sending connection context
0808 02:20:47.793260 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 02:20:47.793639 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":154,"threads_started":1}
I20260808 02:20:47.796089 11682 heartbeater.cc:347] Connected to a master server at 127.24.79.254:44707
I20260808 02:20:47.796418 11682 heartbeater.cc:464] Registering TS with master...
I20260808 02:20:47.797187 11682 heartbeater.cc:511] Master 127.24.79.254:44707 requested a full tablet report, sending...
W20260808 02:20:47.799453 11682 heartbeater.cc:644] Failed to heartbeat to 127.24.79.254:44707 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 02:20:47.799628 11682 heartbeater.cc:464] Registering TS with master...
I20260808 02:20:47.799916 11682 heartbeater.cc:511] Master 127.24.79.254:44707 requested a full tablet report, sending...
I20260808 02:20:47.801059 11682 heartbeater.cc:464] Registering TS with master...
I20260808 02:20:47.801297 11682 heartbeater.cc:511] Master 127.24.79.254:44707 requested a full tablet report, sending...
W20260808 02:20:47.802286 11682 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:20:47.966399 11685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:20:47.966758 11685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:20:47.966825 11685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:20:47.971979 11685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:20:47.972080 11685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:20:47.972167 11685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:20:47.979669 11685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:41439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.79.212:60923/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:42173
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=39037
--tserver_master_addrs=127.24.79.254:44707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:20:47.981124 11685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:20:47.982960 11685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:20:47.997303 11692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:47.997416 11693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:47.997339 11696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:47.998514 11685 server_base.cc:1034] running on GCE node
I20260808 02:20:47.999202 11685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:20:48.000416 11685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:20:48.001679 11685 hybrid_clock.cc:648] HybridClock initialized: now 1786155648001648 us; error 55 us; skew 500 ppm
Aug 08 02:20:48 dist-test-slave-1vmg krb5kdc[9808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155648, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:20:48.006202 11685 init.cc:377] Logged in from keytab as kudu/127.24.79.195@KRBTEST.COM (short username kudu)
I20260808 02:20:48.007836 11685 webserver.cc:460] Webserver started at http://127.24.79.195:39037/ using document root <none> and password file <none>
I20260808 02:20:48.008723 11685 fs_manager.cc:359] Metadata directory not provided
I20260808 02:20:48.008858 11685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 02:20:48 dist-test-slave-1vmg krb5kdc[9808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155648, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for HTTP/127.24.79.212@KRBTEST.COM
I20260808 02:20:48.032109 11685 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.003s sys 0.000s
I20260808 02:20:48.035276 11707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:48.037348 11685 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:20:48.037530 11685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "cdbd424f32904948932ab70a1648a407"
format_stamp: "Formatted at 2026-08-08 02:20:46 on dist-test-slave-1vmg"
server_key: "b2683c7976ce60ab373124b488b8ce8d"
server_key_iv: "39b2253df6b8c89d410853d509af7828"
server_key_version: "kuduclusterkey@0"
I20260808 02:20:48.038218 11685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:20:48.063532 11685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:20:48.067461 11685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:20:48.067692 11685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:20:48.068424 11685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:20:48.069454 11685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:20:48.069511 11685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:48.069572 11685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:20:48.069598 11685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:48.085932 11685 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:42173
I20260808 02:20:48.086035 11866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:42173 every 8 connection(s)
I20260808 02:20:48.087680 11685 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:20:48.089063 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 11685
I20260808 02:20:48.110862 11506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:48.091954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33394 (local address 127.24.79.254:44707)
0808 02:20:48.092104 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:48.092108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:48.092944 (+ 836us) server_negotiation.cc:407] Connection header received
0808 02:20:48.093074 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:48.093077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:48.093139 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:48.093255 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:48.095991 (+ 2736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:48.096855 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:48.099899 (+ 3044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:48.101370 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:48.101477 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:48.103693 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:48.103698 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:48.103701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:48.103747 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:48.105703 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:48.107142 (+ 1439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:48.107147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:48.107149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:48.107384 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:48.109700 (+ 2316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:48.109705 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:48.109707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:48.109952 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:48.110309 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:48.110486 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 02:20:48.110666 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":997,"server-negotiator.queue_time_us":67}
Aug 08 02:20:48 dist-test-slave-1vmg krb5kdc[9808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155648, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:20:48.121496 11875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:48.092266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:34587 (local address 127.24.79.254:44707)
0808 02:20:48.092529 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:48.092533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:48.092551 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:20:48.096459 (+ 3908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:48.096463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:48.096527 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:48.096593 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:48.099963 (+ 3370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:48.100896 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:48.109018 (+ 8122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:48.109445 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:48.109534 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:48.116166 (+ 6632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:48.116171 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:48.116174 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:48.116212 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:48.118432 (+ 2220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:48.119363 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:48.119368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:48.119370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:48.119488 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:48.119996 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:48.120000 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:48.120001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:48.120260 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:48.120371 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:48.121089 (+ 718us) server_negotiation.cc:299] Negotiation successful
0808 02:20:48.121266 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":201,"thread_start_us":98,"threads_started":1}
I20260808 02:20:48.122503 11874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:48.090730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:44707 (local address 127.24.79.195:34587)
0808 02:20:48.091335 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:48.091399 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:48.096117 (+ 4718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:48.097113 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:48.097124 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:48.097574 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:20:48.098352 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:48.098376 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:48.106456 (+ 8080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:48.106460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:48.108813 (+ 2353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:48.108835 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:48.112681 (+ 3846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:48.112694 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:48.112981 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:48.113856 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:20:48.113878 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:20:48.115858 (+ 1980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:20:48.118626 (+ 2768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:48.118636 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:48.118660 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:48.119118 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:48.119613 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:48.119617 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:48.119620 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:48.119861 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:48.120396 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:20:48.120406 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:20:48.120756 (+ 350us) client_negotiation.cc:769] Sending connection context
0808 02:20:48.120979 (+ 223us) client_negotiation.cc:240] Negotiation successful
0808 02:20:48.121397 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":201,"threads_started":1}
I20260808 02:20:48.124437 11867 heartbeater.cc:347] Connected to a master server at 127.24.79.254:44707
I20260808 02:20:48.124752 11867 heartbeater.cc:464] Registering TS with master...
I20260808 02:20:48.125514 11867 heartbeater.cc:511] Master 127.24.79.254:44707 requested a full tablet report, sending...
W20260808 02:20:48.127681 11867 heartbeater.cc:644] Failed to heartbeat to 127.24.79.254:44707 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 02:20:48.127836 11867 heartbeater.cc:464] Registering TS with master...
I20260808 02:20:48.128070 11867 heartbeater.cc:511] Master 127.24.79.254:44707 requested a full tablet report, sending...
I20260808 02:20:48.129242 11867 heartbeater.cc:464] Registering TS with master...
I20260808 02:20:48.129500 11867 heartbeater.cc:511] Master 127.24.79.254:44707 requested a full tablet report, sending...
W20260808 02:20:48.130666 11867 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 02:20:48.550586 11497 heartbeater.cc:464] Registering TS with master...
I20260808 02:20:48.550999 11497 heartbeater.cc:511] Master 127.24.79.254:44707 requested a full tablet report, sending...
I20260808 02:20:48.552462 11226 ts_manager.cc:194] Registered new tserver with Master: e94c5ae1175943099a58d0b426d0472a (127.24.79.193:45865)
I20260808 02:20:48.554335 11226 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.193@KRBTEST.COM'} at 127.24.79.193:36515
Aug 08 02:20:48 dist-test-slave-1vmg krb5kdc[9808](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786155648, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:20:48.802570 11682 heartbeater.cc:464] Registering TS with master...
I20260808 02:20:48.802984 11682 heartbeater.cc:511] Master 127.24.79.254:44707 requested a full tablet report, sending...
I20260808 02:20:48.803999 11226 ts_manager.cc:194] Registered new tserver with Master: 5eb44967e0264963918977ccd1f2b9be (127.24.79.194:37583)
I20260808 02:20:48.804677 11226 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.194@KRBTEST.COM'} at 127.24.79.194:38357
Aug 08 02:20:49 dist-test-slave-1vmg krb5kdc[9808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155648, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for HTTP/127.24.79.212@KRBTEST.COM
I20260808 02:20:49.131043 11867 heartbeater.cc:464] Registering TS with master...
I20260808 02:20:49.131448 11867 heartbeater.cc:511] Master 127.24.79.254:44707 requested a full tablet report, sending...
I20260808 02:20:49.132855 11226 ts_manager.cc:194] Registered new tserver with Master: cdbd424f32904948932ab70a1648a407 (127.24.79.195:42173)
I20260808 02:20:49.134284 11226 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.195@KRBTEST.COM'} at 127.24.79.195:34587
I20260808 02:20:49.135854 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:20:49.149755 11897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:49.137919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33408 (local address 127.24.79.254:44707)
0808 02:20:49.138374 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:49.138379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:49.138401 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:20:49.138531 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:49.138533 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:49.138587 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:49.138694 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:49.139683 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:49.140584 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:49.141918 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:49.142314 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:49.142422 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:49.144644 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:49.144649 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:49.144652 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:49.144683 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:49.146757 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:49.147454 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:49.147458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:49.147460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:49.147884 (+ 424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:49.148236 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:49.148240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:49.148242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:49.148510 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:49.148653 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:49.149396 (+ 743us) server_negotiation.cc:299] Negotiation successful
0808 02:20:49.149558 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":113,"threads_started":1}
I20260808 02:20:49.155287 11226 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33408:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:20:49.161068 11226 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:20:49.183913 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:49.175847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:42173 (local address 127.0.0.1:60660)
0808 02:20:49.176650 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:49.176724 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:49.176891 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:49.178415 (+ 1524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:49.178423 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:49.178456 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:20:49.178635 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:49.178644 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:49.180418 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:49.180422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:49.181888 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:49.181901 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:49.183247 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:49.183252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:49.183391 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:49.183446 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 02:20:49.183568 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 02:20:49.183730 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":141,"threads_started":1}
I20260808 02:20:49.184958 11904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:49.176016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60660 (local address 127.24.79.195:42173)
0808 02:20:49.177263 (+ 1247us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:49.177272 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:49.177304 (+ 32us) server_negotiation.cc:407] Connection header received
0808 02:20:49.177396 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:49.177402 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:49.178178 (+ 776us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:49.178831 (+ 653us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:20:49.179022 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:49.180260 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:49.182071 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:49.183067 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:49.184411 (+ 1344us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:49.184607 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:49.184672 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 02:20:49.184797 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1100,"thread_start_us":93,"threads_started":1}
I20260808 02:20:49.188668 11907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:49.177382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:37583 (local address 127.0.0.1:47028)
0808 02:20:49.180211 (+ 2829us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:49.180227 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:49.180325 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:49.181322 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:49.181325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:49.181347 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:20:49.181460 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:49.181468 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:49.184838 (+ 3370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:49.184841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:49.186252 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:49.186264 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:49.188179 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:49.188183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:49.188293 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:49.188323 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 02:20:49.188403 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 02:20:49.188494 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2757,"thread_start_us":788,"threads_started":1}
I20260808 02:20:49.189528 11906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:49.177935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47028 (local address 127.24.79.194:37583)
0808 02:20:49.180599 (+ 2664us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:49.180607 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:49.180637 (+ 30us) server_negotiation.cc:407] Connection header received
0808 02:20:49.180721 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:49.180739 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:49.180990 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:49.181153 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:20:49.181917 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:49.184675 (+ 2758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:49.186421 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:49.187795 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:49.187912 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:49.188087 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:49.189249 (+ 1162us) server_negotiation.cc:299] Negotiation successful
0808 02:20:49.189379 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2534,"thread_start_us":73,"threads_started":1}
I20260808 02:20:49.189208 11767 tablet_service.cc:1467] Processing CreateTablet for tablet 9d985e17fe7345309409e71de40d4d11 (DEFAULT_TABLE table=test-table [id=5d2a86d58801408c8b854c08aa7f1fb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:20:49.193778 11568 tablet_service.cc:1467] Processing CreateTablet for tablet 9d985e17fe7345309409e71de40d4d11 (DEFAULT_TABLE table=test-table [id=5d2a86d58801408c8b854c08aa7f1fb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:20:49.195344 11568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d985e17fe7345309409e71de40d4d11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:20:49.196280 11905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:49.178470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:45865 (local address 127.0.0.1:59576)
0808 02:20:49.181700 (+ 3230us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:49.181713 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:49.181802 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:49.183891 (+ 2089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:49.183894 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:49.183911 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:20:49.183998 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:49.184004 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:49.185351 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:49.185354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:49.186435 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:49.186444 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:49.195720 (+ 9276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:49.195725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:49.195844 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:49.195875 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 02:20:49.195954 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 02:20:49.196049 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3162,"thread_start_us":89,"threads_started":1}
I20260808 02:20:49.196755 11767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d985e17fe7345309409e71de40d4d11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:20:49.196812 11909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:49.179364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59576 (local address 127.24.79.193:45865)
0808 02:20:49.183163 (+ 3799us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:49.183171 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:49.183199 (+ 28us) server_negotiation.cc:407] Connection header received
0808 02:20:49.183296 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:49.183303 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:49.183563 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:49.183733 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:20:49.184135 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:49.185208 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:49.194106 (+ 8898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:49.195285 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:49.195454 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:49.195620 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:49.196534 (+ 914us) server_negotiation.cc:299] Negotiation successful
0808 02:20:49.196669 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3672,"thread_start_us":131,"threads_started":1}
I20260808 02:20:49.200711 11410 tablet_service.cc:1467] Processing CreateTablet for tablet 9d985e17fe7345309409e71de40d4d11 (DEFAULT_TABLE table=test-table [id=5d2a86d58801408c8b854c08aa7f1fb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:20:49.202229 11410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d985e17fe7345309409e71de40d4d11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:20:49.221654 11911 tablet_bootstrap.cc:492] T 9d985e17fe7345309409e71de40d4d11 P cdbd424f32904948932ab70a1648a407: Bootstrap starting.
I20260808 02:20:49.225931 11911 tablet_bootstrap.cc:654] T 9d985e17fe7345309409e71de40d4d11 P cdbd424f32904948932ab70a1648a407: Neither blocks nor log segments found. Creating new log.
I20260808 02:20:49.227118 11911 log.cc:824] T 9d985e17fe7345309409e71de40d4d11 P cdbd424f32904948932ab70a1648a407: Log is configured to *not* fsync() on all Append() calls
I20260808 02:20:49.229362 11912 tablet_bootstrap.cc:492] T 9d985e17fe7345309409e71de40d4d11 P e94c5ae1175943099a58d0b426d0472a: Bootstrap starting.
I20260808 02:20:49.230499 11911 tablet_bootstrap.cc:492] T 9d985e17fe7345309409e71de40d4d11 P cdbd424f32904948932ab70a1648a407: No bootstrap required, opened a new log
I20260808 02:20:49.230767 11911 ts_tablet_manager.cc:1397] T 9d985e17fe7345309409e71de40d4d11 P cdbd424f32904948932ab70a1648a407: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.004s
I20260808 02:20:49.233017 11912 tablet_bootstrap.cc:654] T 9d985e17fe7345309409e71de40d4d11 P e94c5ae1175943099a58d0b426d0472a: Neither blocks nor log segments found. Creating new log.
I20260808 02:20:49.234421 11912 log.cc:824] T 9d985e17fe7345309409e71de40d4d11 P e94c5ae1175943099a58d0b426d0472a: Log is configured to *not* fsync() on all Append() calls
I20260808 02:20:49.235438 11913 tablet_bootstrap.cc:492] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be: Bootstrap starting.
I20260808 02:20:49.237365 11911 raft_consensus.cc:348] T 9d985e17fe7345309409e71de40d4d11 P cdbd424f32904948932ab70a1648a407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbd424f32904948932ab70a1648a407" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 42173 } } peers { permanent_uuid: "e94c5ae1175943099a58d0b426d0472a" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45865 } } peers { permanent_uuid: "5eb44967e0264963918977ccd1f2b9be" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37583 } }
I20260808 02:20:49.237716 11911 raft_consensus.cc:374] T 9d985e17fe7345309409e71de40d4d11 P cdbd424f32904948932ab70a1648a407 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:20:49.237777 11911 raft_consensus.cc:729] T 9d985e17fe7345309409e71de40d4d11 P cdbd424f32904948932ab70a1648a407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdbd424f32904948932ab70a1648a407, State: Initialized, Role: FOLLOWER
I20260808 02:20:49.238061 11912 tablet_bootstrap.cc:492] T 9d985e17fe7345309409e71de40d4d11 P e94c5ae1175943099a58d0b426d0472a: No bootstrap required, opened a new log
I20260808 02:20:49.238304 11912 ts_tablet_manager.cc:1397] T 9d985e17fe7345309409e71de40d4d11 P e94c5ae1175943099a58d0b426d0472a: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 02:20:49.238368 11913 tablet_bootstrap.cc:654] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be: Neither blocks nor log segments found. Creating new log.
I20260808 02:20:49.238358 11911 consensus_queue.cc:260] T 9d985e17fe7345309409e71de40d4d11 P cdbd424f32904948932ab70a1648a407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbd424f32904948932ab70a1648a407" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 42173 } } peers { permanent_uuid: "e94c5ae1175943099a58d0b426d0472a" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45865 } } peers { permanent_uuid: "5eb44967e0264963918977ccd1f2b9be" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37583 } }
I20260808 02:20:49.239377 11913 log.cc:824] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be: Log is configured to *not* fsync() on all Append() calls
I20260808 02:20:49.239729 11911 ts_tablet_manager.cc:1428] T 9d985e17fe7345309409e71de40d4d11 P cdbd424f32904948932ab70a1648a407: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260808 02:20:49.240218 11867 heartbeater.cc:503] Master 127.24.79.254:44707 was elected leader, sending a full tablet report...
I20260808 02:20:49.243470 11912 raft_consensus.cc:348] T 9d985e17fe7345309409e71de40d4d11 P e94c5ae1175943099a58d0b426d0472a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbd424f32904948932ab70a1648a407" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 42173 } } peers { permanent_uuid: "e94c5ae1175943099a58d0b426d0472a" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45865 } } peers { permanent_uuid: "5eb44967e0264963918977ccd1f2b9be" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37583 } }
I20260808 02:20:49.244138 11912 raft_consensus.cc:374] T 9d985e17fe7345309409e71de40d4d11 P e94c5ae1175943099a58d0b426d0472a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:20:49.244268 11912 raft_consensus.cc:729] T 9d985e17fe7345309409e71de40d4d11 P e94c5ae1175943099a58d0b426d0472a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e94c5ae1175943099a58d0b426d0472a, State: Initialized, Role: FOLLOWER
I20260808 02:20:49.244939 11912 consensus_queue.cc:260] T 9d985e17fe7345309409e71de40d4d11 P e94c5ae1175943099a58d0b426d0472a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbd424f32904948932ab70a1648a407" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 42173 } } peers { permanent_uuid: "e94c5ae1175943099a58d0b426d0472a" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45865 } } peers { permanent_uuid: "5eb44967e0264963918977ccd1f2b9be" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37583 } }
I20260808 02:20:49.246430 11912 ts_tablet_manager.cc:1428] T 9d985e17fe7345309409e71de40d4d11 P e94c5ae1175943099a58d0b426d0472a: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260808 02:20:49.247653 11497 heartbeater.cc:503] Master 127.24.79.254:44707 was elected leader, sending a full tablet report...
I20260808 02:20:49.250969 11913 tablet_bootstrap.cc:492] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be: No bootstrap required, opened a new log
I20260808 02:20:49.251284 11913 ts_tablet_manager.cc:1397] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.000s
W20260808 02:20:49.251588 11504 tablet.cc:2369] T 9d985e17fe7345309409e71de40d4d11 P e94c5ae1175943099a58d0b426d0472a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:20:49.256397 11913 raft_consensus.cc:348] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbd424f32904948932ab70a1648a407" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 42173 } } peers { permanent_uuid: "e94c5ae1175943099a58d0b426d0472a" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45865 } } peers { permanent_uuid: "5eb44967e0264963918977ccd1f2b9be" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37583 } }
I20260808 02:20:49.256850 11913 raft_consensus.cc:374] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:20:49.256924 11913 raft_consensus.cc:729] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eb44967e0264963918977ccd1f2b9be, State: Initialized, Role: FOLLOWER
I20260808 02:20:49.257611 11913 consensus_queue.cc:260] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbd424f32904948932ab70a1648a407" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 42173 } } peers { permanent_uuid: "e94c5ae1175943099a58d0b426d0472a" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45865 } } peers { permanent_uuid: "5eb44967e0264963918977ccd1f2b9be" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37583 } }
I20260808 02:20:49.258513 11682 heartbeater.cc:503] Master 127.24.79.254:44707 was elected leader, sending a full tablet report...
I20260808 02:20:49.259078 11913 ts_tablet_manager.cc:1428] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
W20260808 02:20:49.277876 11683 tablet.cc:2369] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:20:49.341231 11870 tablet.cc:2369] T 9d985e17fe7345309409e71de40d4d11 P cdbd424f32904948932ab70a1648a407: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 02:20:49 dist-test-slave-1vmg krb5kdc[9808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155648, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for HTTP/127.24.79.212@KRBTEST.COM
I20260808 02:20:49.490347 11920 raft_consensus.cc:482] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:20:49.490638 11920 raft_consensus.cc:504] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbd424f32904948932ab70a1648a407" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 42173 } } peers { permanent_uuid: "e94c5ae1175943099a58d0b426d0472a" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45865 } } peers { permanent_uuid: "5eb44967e0264963918977ccd1f2b9be" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37583 } }
I20260808 02:20:49.492399 11920 leader_election.cc:288] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdbd424f32904948932ab70a1648a407 (127.24.79.195:42173), e94c5ae1175943099a58d0b426d0472a (127.24.79.193:45865)
I20260808 02:20:49.500504 11909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:49.493681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:37437 (local address 127.24.79.193:45865)
0808 02:20:49.493830 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:49.493835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:49.496504 (+ 2669us) server_negotiation.cc:407] Connection header received
0808 02:20:49.496682 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:49.496685 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:49.496731 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:49.496775 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:20:49.497229 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:49.497808 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:49.499018 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:49.499641 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:49.499800 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:49.499864 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:49.500247 (+ 383us) server_negotiation.cc:299] Negotiation successful
0808 02:20:49.500348 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 02:20:49.501274 11924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:49.493176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:45865 (local address 127.24.79.194:37437)
0808 02:20:49.496405 (+ 3229us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:49.496421 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:49.496573 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:49.496901 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:49.496904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:49.496919 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:20:49.497047 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:49.497053 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:49.497958 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:49.497960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:49.498840 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:49.498850 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:49.499947 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:49.499951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:49.500085 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:49.500113 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 02:20:49.500999 (+ 886us) client_negotiation.cc:240] Negotiation successful
0808 02:20:49.501092 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3151,"thread_start_us":80,"threads_started":1}
I20260808 02:20:49.503135 11904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:49.492881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:42673 (local address 127.24.79.195:42173)
0808 02:20:49.493490 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:49.493496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:49.495135 (+ 1639us) server_negotiation.cc:407] Connection header received
0808 02:20:49.495263 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:49.495267 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:49.495328 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:49.495427 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:20:49.499111 (+ 3684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:49.499859 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:49.501287 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:49.501827 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:49.501946 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:49.502002 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:49.502826 (+ 824us) server_negotiation.cc:299] Negotiation successful
0808 02:20:49.502985 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498}
I20260808 02:20:49.503670 11925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:49.492695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:42173 (local address 127.24.79.194:42673)
0808 02:20:49.494979 (+ 2284us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:49.494999 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:49.495153 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:49.495903 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:49.495907 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:49.495936 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:20:49.496102 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:49.496109 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:49.500007 (+ 3898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:49.500010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:49.501106 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:49.501117 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:49.502520 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:49.502524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:49.502643 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:49.502672 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 02:20:49.503393 (+ 721us) client_negotiation.cc:240] Negotiation successful
0808 02:20:49.503516 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2166,"thread_start_us":120,"threads_started":1}
I20260808 02:20:49.504505 11802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d985e17fe7345309409e71de40d4d11" candidate_uuid: "5eb44967e0264963918977ccd1f2b9be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdbd424f32904948932ab70a1648a407" is_pre_election: true
I20260808 02:20:49.504923 11802 raft_consensus.cc:2454] T 9d985e17fe7345309409e71de40d4d11 P cdbd424f32904948932ab70a1648a407 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5eb44967e0264963918977ccd1f2b9be in term 0.
I20260808 02:20:49.505626 11529 leader_election.cc:302] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5eb44967e0264963918977ccd1f2b9be, cdbd424f32904948932ab70a1648a407; no voters:
I20260808 02:20:49.506070 11920 raft_consensus.cc:2781] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:20:49.506191 11920 raft_consensus.cc:482] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:20:49.506232 11920 raft_consensus.cc:3037] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:20:49.506846 11438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d985e17fe7345309409e71de40d4d11" candidate_uuid: "5eb44967e0264963918977ccd1f2b9be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e94c5ae1175943099a58d0b426d0472a" is_pre_election: true
I20260808 02:20:49.507537 11438 raft_consensus.cc:2454] T 9d985e17fe7345309409e71de40d4d11 P e94c5ae1175943099a58d0b426d0472a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5eb44967e0264963918977ccd1f2b9be in term 0.
I20260808 02:20:49.507989 11920 raft_consensus.cc:504] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbd424f32904948932ab70a1648a407" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 42173 } } peers { permanent_uuid: "e94c5ae1175943099a58d0b426d0472a" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45865 } } peers { permanent_uuid: "5eb44967e0264963918977ccd1f2b9be" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37583 } }
I20260808 02:20:49.508502 11920 leader_election.cc:288] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be [CANDIDATE]: Term 1 election: Requested vote from peers cdbd424f32904948932ab70a1648a407 (127.24.79.195:42173), e94c5ae1175943099a58d0b426d0472a (127.24.79.193:45865)
I20260808 02:20:49.509742 11438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d985e17fe7345309409e71de40d4d11" candidate_uuid: "5eb44967e0264963918977ccd1f2b9be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e94c5ae1175943099a58d0b426d0472a"
I20260808 02:20:49.509742 11802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d985e17fe7345309409e71de40d4d11" candidate_uuid: "5eb44967e0264963918977ccd1f2b9be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdbd424f32904948932ab70a1648a407"
I20260808 02:20:49.509919 11438 raft_consensus.cc:3037] T 9d985e17fe7345309409e71de40d4d11 P e94c5ae1175943099a58d0b426d0472a [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:20:49.509919 11802 raft_consensus.cc:3037] T 9d985e17fe7345309409e71de40d4d11 P cdbd424f32904948932ab70a1648a407 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:20:49.511479 11438 raft_consensus.cc:2454] T 9d985e17fe7345309409e71de40d4d11 P e94c5ae1175943099a58d0b426d0472a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5eb44967e0264963918977ccd1f2b9be in term 1.
I20260808 02:20:49.511479 11802 raft_consensus.cc:2454] T 9d985e17fe7345309409e71de40d4d11 P cdbd424f32904948932ab70a1648a407 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5eb44967e0264963918977ccd1f2b9be in term 1.
I20260808 02:20:49.511943 11529 leader_election.cc:302] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5eb44967e0264963918977ccd1f2b9be, e94c5ae1175943099a58d0b426d0472a; no voters:
I20260808 02:20:49.512161 11920 raft_consensus.cc:2781] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:20:49.512444 11920 raft_consensus.cc:686] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be [term 1 LEADER]: Becoming Leader. State: Replica: 5eb44967e0264963918977ccd1f2b9be, State: Running, Role: LEADER
I20260808 02:20:49.512925 11920 consensus_queue.cc:237] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbd424f32904948932ab70a1648a407" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 42173 } } peers { permanent_uuid: "e94c5ae1175943099a58d0b426d0472a" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45865 } } peers { permanent_uuid: "5eb44967e0264963918977ccd1f2b9be" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37583 } }
I20260808 02:20:49.516978 11226 catalog_manager.cc:5495] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be reported cstate change: term changed from 0 to 1, leader changed from <none> to 5eb44967e0264963918977ccd1f2b9be (127.24.79.194). New cstate: current_term: 1 leader_uuid: "5eb44967e0264963918977ccd1f2b9be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbd424f32904948932ab70a1648a407" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 42173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e94c5ae1175943099a58d0b426d0472a" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5eb44967e0264963918977ccd1f2b9be" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37583 } health_report { overall_health: HEALTHY } } }
I20260808 02:20:49.562357 11906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:49.556833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47036 (local address 127.24.79.194:37583)
0808 02:20:49.556961 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:49.556967 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:49.556985 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:20:49.557232 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:49.557237 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:49.557312 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:49.557435 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:20:49.557716 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:49.558337 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:49.559585 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:49.559956 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:49.560072 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:49.560558 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:20:49.561307 (+ 749us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:20:49.561463 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:49.562051 (+ 588us) server_negotiation.cc:299] Negotiation successful
0808 02:20:49.562141 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:20:49.571111 11802 raft_consensus.cc:1261] T 9d985e17fe7345309409e71de40d4d11 P cdbd424f32904948932ab70a1648a407 [term 1 FOLLOWER]: Refusing update from remote peer 5eb44967e0264963918977ccd1f2b9be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:20:49.572273 11438 raft_consensus.cc:1261] T 9d985e17fe7345309409e71de40d4d11 P e94c5ae1175943099a58d0b426d0472a [term 1 FOLLOWER]: Refusing update from remote peer 5eb44967e0264963918977ccd1f2b9be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:20:49.572866 11920 consensus_queue.cc:1035] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdbd424f32904948932ab70a1648a407" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 42173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:20:49.573137 11926 consensus_queue.cc:1035] T 9d985e17fe7345309409e71de40d4d11 P 5eb44967e0264963918977ccd1f2b9be [LEADER]: Connected to new peer: Peer: permanent_uuid: "e94c5ae1175943099a58d0b426d0472a" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 45865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:20:49.584012 11934 mvcc.cc:205] Tried to move back new op lower bound from 7316093540632883200 to 7316093540408213504. Current Snapshot: MvccSnapshot[applied={T|T < 7316093540632883200}]
I20260808 02:20:49.586349 11930 mvcc.cc:205] Tried to move back new op lower bound from 7316093540632883200 to 7316093540408213504. Current Snapshot: MvccSnapshot[applied={T|T < 7316093540632883200}]
I20260808 02:20:49.599783 11937 mvcc.cc:205] Tried to move back new op lower bound from 7316093540632883200 to 7316093540408213504. Current Snapshot: MvccSnapshot[applied={T|T < 7316093540632883200}]
I20260808 02:20:49.601217 11226 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:20:49.601425 11226 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:20:49.604856 11226 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b187e43a949f4677a8f14737b32cabb9: Sending DeleteTablet for 3 replicas of tablet 9d985e17fe7345309409e71de40d4d11
I20260808 02:20:49.606280 11410 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d985e17fe7345309409e71de40d4d11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:20:49 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:59576
I20260808 02:20:49.607061 11948 tablet_replica.cc:330] T 9d985e17fe7345309409e71de40d4d11 P e94c5ae1175943099a58d0b426d0472a: stopping tablet replica
I20260808 02:20:49.607077 11568 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d985e17fe7345309409e71de40d4d11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:20:49 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:47028
I20260808 02:20:49.607686 11948 raft_consensus.cc:2229] T 9d985e17fe7345309409e71de40d4d11 P e94c5ae1175943099a58d0b426d0472a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:20:49.608194 11948 raft_consensus.cc:2258] T 9d985e17fe7345309409e71de40d4d11 P e94c5ae1175943099a58d0b426d0472a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:20:49.608464 11767 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d985e17fe7345309409e71de40d4d11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:20:49 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:60660
I20260808 02:20:49.609489 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 11284
I20260808 02:20:49.619676 11949 tablet_replica.cc:330] T 9d985e17fe7345309409e71de40d4d11 P cdbd424f32904948932ab70a1648a407: stopping tablet replica
I20260808 02:20:49.620296 11949 raft_consensus.cc:2229] T 9d985e17fe7345309409e71de40d4d11 P cdbd424f32904948932ab70a1648a407 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:20:49.620922 11949 raft_consensus.cc:2258] T 9d985e17fe7345309409e71de40d4d11 P cdbd424f32904948932ab70a1648a407 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:20:49.624248 11949 ts_tablet_manager.cc:1905] T 9d985e17fe7345309409e71de40d4d11 P cdbd424f32904948932ab70a1648a407: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:20:49.628130 11949 ts_tablet_manager.cc:1918] T 9d985e17fe7345309409e71de40d4d11 P cdbd424f32904948932ab70a1648a407: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:20:49.628413 11949 log.cc:1196] T 9d985e17fe7345309409e71de40d4d11 P cdbd424f32904948932ab70a1648a407: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/9d985e17fe7345309409e71de40d4d11
I20260808 02:20:49.628980 11949 ts_tablet_manager.cc:1939] T 9d985e17fe7345309409e71de40d4d11 P cdbd424f32904948932ab70a1648a407: Deleting consensus metadata
I20260808 02:20:49.630950 11203 catalog_manager.cc:4841] TS cdbd424f32904948932ab70a1648a407 (127.24.79.195:42173): tablet 9d985e17fe7345309409e71de40d4d11 (table test-table [id=5d2a86d58801408c8b854c08aa7f1fb2]) successfully deleted
W20260808 02:20:49.631338 11203 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.79.193:45865 (error 108)
I20260808 02:20:49.631548 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 11499
I20260808 02:20:49.644932 11905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:49.644544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:45865 (local address 127.0.0.1:59586)
0808 02:20:49.644706 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:49.644808 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.193:45865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260808 02:20:49.645318 11203 catalog_manager.cc:4584] TS e94c5ae1175943099a58d0b426d0472a (127.24.79.193:45865): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9d985e17fe7345309409e71de40d4d11: Network error: Client connection negotiation failed: client connection to 127.24.79.193:45865: connect: Connection refused (error 111)
I20260808 02:20:49.653193 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 11685
I20260808 02:20:49.661955 11905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:49.653250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:37583 (local address 127.0.0.1:47040)
0808 02:20:49.661741 (+ 8491us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:49.661831 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.194:37583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8352}
I20260808 02:20:49.670490 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 11177
I20260808 02:20:49.697402 24895 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 02:20:50.044737 24895 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 02:20:50.044871 24895 mini_ranger.cc:66] Stopping Ranger...
I20260808 02:20:50.402097 24895 mini_ranger.cc:68] Stopped Ranger
2026-08-08 02:20:50.402 UTC [9846] LOG: received smart shutdown request
2026-08-08 02:20:50.409 UTC [9846] LOG: background worker "logical replication launcher" (PID 9863) exited with exit code 1
2026-08-08 02:20:50.409 UTC [9851] LOG: shutting down
2026-08-08 02:20:50.690 UTC [9846] LOG: database system is shut down
2026-08-08T02:20:50Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (61899 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:20:50 dist-test-slave-1vmg krb5kdc[11977](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:20:50 dist-test-slave-1vmg krb5kdc[11977](info): set up 2 sockets
Aug 08 02:20:50 dist-test-slave-1vmg krb5kdc[11977](info): commencing operation
krb5kdc: starting...
W20260808 02:20:51.970135 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:20:52 dist-test-slave-1vmg krb5kdc[11977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:20:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:20:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:20:52.120230 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:46207
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:46207,127.24.79.253:40605,127.24.79.252:41763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:20:52.246852 11993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:20:52.247248 11993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:20:52.247311 11993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:20:52.251590 11993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:20:52.251679 11993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:20:52.251708 11993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:20:52.251730 11993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:20:52.251749 11993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:20:52.257660 11993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:46207,127.24.79.253:40605,127.24.79.252:41763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:46207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:20:52.258992 11993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:20:52.260167 11993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:20:52.270143 12007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:52.270200 11993 server_base.cc:1034] running on GCE node
W20260808 02:20:52.270159 12002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:52.270147 12003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:52.271271 11993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:20:52.272583 11993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:20:52.273813 11993 hybrid_clock.cc:648] HybridClock initialized: now 1786155652273793 us; error 37 us; skew 500 ppm
Aug 08 02:20:52 dist-test-slave-1vmg krb5kdc[11977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155652, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:20:52.277205 11993 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:20:52.278342 11993 webserver.cc:460] Webserver started at http://127.24.79.254:41557/ using document root <none> and password file <none>
I20260808 02:20:52.279065 11993 fs_manager.cc:359] Metadata directory not provided
I20260808 02:20:52.279142 11993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:20:52.279378 11993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:20:52.281668 11993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "4bc42ebef0e2436ba840ed71ccb17f37"
format_stamp: "Formatted at 2026-08-08 02:20:52 on dist-test-slave-1vmg"
server_key: "9d34458ba0e32ee767d455e8055b1f5a"
server_key_iv: "558a26117171200e3287674f2cb0a7fc"
server_key_version: "encryptionkey@0"
I20260808 02:20:52.282291 11993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "4bc42ebef0e2436ba840ed71ccb17f37"
format_stamp: "Formatted at 2026-08-08 02:20:52 on dist-test-slave-1vmg"
server_key: "9d34458ba0e32ee767d455e8055b1f5a"
server_key_iv: "558a26117171200e3287674f2cb0a7fc"
server_key_version: "encryptionkey@0"
I20260808 02:20:52.286628 11993 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 02:20:52.289510 12020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:52.291148 11993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:20:52.291313 11993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "4bc42ebef0e2436ba840ed71ccb17f37"
format_stamp: "Formatted at 2026-08-08 02:20:52 on dist-test-slave-1vmg"
server_key: "9d34458ba0e32ee767d455e8055b1f5a"
server_key_iv: "558a26117171200e3287674f2cb0a7fc"
server_key_version: "encryptionkey@0"
I20260808 02:20:52.291447 11993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:20:52.310633 11993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:20:52.314182 11993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:20:52.314383 11993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:20:52.322364 11993 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:46207
I20260808 02:20:52.322427 12082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:46207 every 8 connection(s)
I20260808 02:20:52.323555 11993 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:20:52.326975 12083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:20:52.329691 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 11993
I20260808 02:20:52.329810 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:20:52.330082 24895 external_mini_cluster.cc:1428] Setting key b71e6fa18ac904cd4dfe7fc22f713570
I20260808 02:20:52.331497 12083 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46207 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.79.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.253@KRBTEST.COM" created.
Aug 08 02:20:52 dist-test-slave-1vmg krb5kdc[11977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155652, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:20:52.350858 12086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.334600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54492 (local address 127.24.79.254:46207)
0808 02:20:52.335321 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:52.335340 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:52.335543 (+ 203us) server_negotiation.cc:407] Connection header received
0808 02:20:52.336984 (+ 1441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:52.336989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:52.337196 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:52.337358 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:52.338508 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.339568 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.340802 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.341300 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.341457 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:52.345345 (+ 3888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:52.345351 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:52.345367 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:52.345394 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:52.347224 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.347796 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.347802 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.347803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.347937 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.348266 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.348269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.348270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.348746 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:52.348891 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:52.349310 (+ 419us) server_negotiation.cc:299] Negotiation successful
0808 02:20:52.349586 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":580,"spinlock_wait_cycles":73984,"thread_start_us":245,"threads_started":1}
I20260808 02:20:52.350858 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.334600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:46207 (local address 127.0.0.1:54492)
0808 02:20:52.335337 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.335404 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:52.336602 (+ 1198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:52.337420 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:52.337427 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:52.337912 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:20:52.338319 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:52.338341 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.339708 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.339712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:52.340657 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:52.340669 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.341500 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.341504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:52.341722 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:52.342427 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:20:52.342447 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:20:52.345148 (+ 2701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:20:52.347362 (+ 2214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:52.347373 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:52.347387 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:52.347671 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:52.348037 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:52.348040 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:52.348042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:52.348172 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:52.348873 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:20:52.348880 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:20:52.349038 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 02:20:52.349220 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 02:20:52.349659 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":175,"thread_start_us":223,"threads_started":1}
I20260808 02:20:52.354338 12083 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:20:52.355443 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.355000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:40605 (local address 127.0.0.1:43610)
0808 02:20:52.355241 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.355349 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.253:40605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 02:20:52.355736 12024 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.79.253:40605: connect: Connection refused (error 111)
I20260808 02:20:52.356954 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.356673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:40605 (local address 127.0.0.1:43624)
0808 02:20:52.356817 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.356874 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.253:40605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 02:20:52.357259 12083 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.253:40605: Network error: Client connection negotiation failed: client connection to 127.24.79.253:40605: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.253.keytab.
Entry for principal kudu/127.24.79.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.253.keytab.
I20260808 02:20:52.368046 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.253:40605
--webserver_interface=127.24.79.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.253.keytab
--principal=kudu/127.24.79.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:46207,127.24.79.253:40605,127.24.79.252:41763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:20:52.406701 12083 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } attempt: 1
I20260808 02:20:52.407743 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.407409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:40605 (local address 127.0.0.1:43638)
0808 02:20:52.407583 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.407656 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.253:40605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 02:20:52.408499 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.408265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:40605 (local address 127.0.0.1:43644)
0808 02:20:52.408382 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.408430 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.253:40605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 02:20:52.408631 12083 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.253:40605: Network error: Client connection negotiation failed: client connection to 127.24.79.253:40605: connect: Connection refused (error 111)
I20260808 02:20:52.476859 12083 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } attempt: 2
I20260808 02:20:52.477877 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.477549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:40605 (local address 127.0.0.1:43656)
0808 02:20:52.477720 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.477790 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.253:40605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 02:20:52.478695 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.478446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:40605 (local address 127.0.0.1:43672)
0808 02:20:52.478577 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.478626 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.253:40605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 02:20:52.478819 12083 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.253:40605: Network error: Client connection negotiation failed: client connection to 127.24.79.253:40605: connect: Connection refused (error 111)
W20260808 02:20:52.493829 12091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:20:52.494184 12091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:20:52.494249 12091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:20:52.498649 12091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:20:52.498746 12091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:20:52.498780 12091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:20:52.498800 12091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:20:52.498822 12091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:20:52.505033 12091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:46207,127.24.79.253:40605,127.24.79.252:41763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.253.keytab
--principal=kudu/127.24.79.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.253:40605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.79.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:20:52.506403 12091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:20:52.507789 12091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:20:52.517742 12091 server_base.cc:1034] running on GCE node
W20260808 02:20:52.517732 12105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:52.517750 12103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:52.517766 12102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:52.518698 12091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:20:52.520056 12091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:20:52.521293 12091 hybrid_clock.cc:648] HybridClock initialized: now 1786155652521269 us; error 42 us; skew 500 ppm
Aug 08 02:20:52 dist-test-slave-1vmg krb5kdc[11977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155652, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:20:52.524842 12091 init.cc:377] Logged in from keytab as kudu/127.24.79.253@KRBTEST.COM (short username kudu)
I20260808 02:20:52.526098 12091 webserver.cc:460] Webserver started at http://127.24.79.253:43917/ using document root <none> and password file <none>
I20260808 02:20:52.526724 12091 fs_manager.cc:359] Metadata directory not provided
I20260808 02:20:52.526799 12091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:20:52.527094 12091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:20:52.529398 12091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-1/data/instance:
uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1"
format_stamp: "Formatted at 2026-08-08 02:20:52 on dist-test-slave-1vmg"
server_key: "c08e9698e2fe51ece6cf9068db72dc90"
server_key_iv: "67ea4e124735680c5b984066668a0570"
server_key_version: "encryptionkey@0"
I20260808 02:20:52.530021 12091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-1/wal/instance:
uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1"
format_stamp: "Formatted at 2026-08-08 02:20:52 on dist-test-slave-1vmg"
server_key: "c08e9698e2fe51ece6cf9068db72dc90"
server_key_iv: "67ea4e124735680c5b984066668a0570"
server_key_version: "encryptionkey@0"
I20260808 02:20:52.534723 12091 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260808 02:20:52.537767 12119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:52.539366 12091 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:20:52.539525 12091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-1/wal
uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1"
format_stamp: "Formatted at 2026-08-08 02:20:52 on dist-test-slave-1vmg"
server_key: "c08e9698e2fe51ece6cf9068db72dc90"
server_key_iv: "67ea4e124735680c5b984066668a0570"
server_key_version: "encryptionkey@0"
I20260808 02:20:52.539662 12091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:20:52.564711 12091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:20:52.568605 12091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:20:52.568863 12091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:20:52.570004 12083 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } attempt: 3
I20260808 02:20:52.570878 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.570579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:40605 (local address 127.0.0.1:43680)
0808 02:20:52.570730 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.570794 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.253:40605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 02:20:52.571646 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.571405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:40605 (local address 127.0.0.1:43696)
0808 02:20:52.571519 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.571573 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.253:40605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 02:20:52.571805 12083 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.253:40605: Network error: Client connection negotiation failed: client connection to 127.24.79.253:40605: connect: Connection refused (error 111)
I20260808 02:20:52.577199 12091 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.253:40605
I20260808 02:20:52.577231 12194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.253:40605 every 8 connection(s)
I20260808 02:20:52.578312 12091 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-1/data/info.pb
I20260808 02:20:52.581722 12196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:20:52.586437 12196 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46207 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:20:52.587508 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 12091
I20260808 02:20:52.587636 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-1/wal/instance
I20260808 02:20:52.587917 24895 external_mini_cluster.cc:1428] Setting key eaa4bcb2c8d47bc6cce5ba42f158f6ba
WARNING: no policy specified for kudu/127.24.79.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.252@KRBTEST.COM" created.
Aug 08 02:20:52 dist-test-slave-1vmg krb5kdc[11977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155652, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.253@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:20:52.608175 12086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.589178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54502 (local address 127.24.79.254:46207)
0808 02:20:52.589359 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:52.589364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:52.590617 (+ 1253us) server_negotiation.cc:407] Connection header received
0808 02:20:52.591766 (+ 1149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:52.591770 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:52.591839 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:52.591943 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:52.593385 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.594206 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.595525 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.595937 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.596059 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:52.603077 (+ 7018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:52.603081 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:52.603084 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:52.603159 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:52.605246 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.606054 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.606059 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.606060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.606180 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.606666 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.606669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.606671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.606980 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:52.607110 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:52.607823 (+ 713us) server_negotiation.cc:299] Negotiation successful
0808 02:20:52.607990 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 02:20:52.609131 12200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.589707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:46207 (local address 127.0.0.1:54502)
0808 02:20:52.590394 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.590461 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:52.591523 (+ 1062us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:52.592120 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:52.592128 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:52.592521 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:20:52.593173 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:52.593198 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.594363 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.594366 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:52.595395 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:52.595408 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.596126 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.596129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:52.596307 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:52.596970 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:20:52.596990 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:20:52.602696 (+ 5706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:20:52.605457 (+ 2761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:52.605466 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:52.605486 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:52.605890 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:52.606331 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:52.606335 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:52.606336 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:52.606537 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:52.607098 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:20:52.607106 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:20:52.607447 (+ 341us) client_negotiation.cc:769] Sending connection context
0808 02:20:52.607685 (+ 238us) client_negotiation.cc:240] Negotiation successful
0808 02:20:52.608025 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":167,"threads_started":1}
I20260808 02:20:52.610772 12196 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:20:52 dist-test-slave-1vmg krb5kdc[11977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155652, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.253@KRBTEST.COM for kudu/127.24.79.253@KRBTEST.COM
I20260808 02:20:52.622179 12200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.611483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:40605 (local address 127.0.0.1:43706)
0808 02:20:52.611666 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.611682 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:52.611779 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:52.612465 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:52.612468 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:52.612651 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:20:52.612768 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:52.612774 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.613873 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.613876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:52.614470 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:52.614480 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.615375 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.615379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:52.615471 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:52.616113 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:20:52.616128 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:20:52.617829 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:20:52.620264 (+ 2435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:52.620267 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:52.620269 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:52.620490 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:52.620882 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:52.620885 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:52.620887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:52.621008 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:52.621671 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:20:52.621673 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:20:52.621783 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 02:20:52.621893 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 02:20:52.622035 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 02:20:52.622376 12202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.611648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43706 (local address 127.24.79.253:40605)
0808 02:20:52.612063 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:52.612074 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:52.612098 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:20:52.612139 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:52.612144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:52.612328 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:52.612443 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:52.612887 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.613738 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.614582 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.615233 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.615330 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:52.618031 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:52.618047 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:52.618067 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:52.618096 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:52.620087 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.620639 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.620647 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.620648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.620751 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.621179 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.621183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.621184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.621533 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:52.621674 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:52.622023 (+ 349us) server_negotiation.cc:299] Negotiation successful
0808 02:20:52.622182 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":198,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.252.keytab.
Entry for principal kudu/127.24.79.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.252.keytab.
I20260808 02:20:52.624756 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.252:41763
--webserver_interface=127.24.79.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.252.keytab
--principal=kudu/127.24.79.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:46207,127.24.79.253:40605,127.24.79.252:41763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:20:52.625124 12196 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:20:52.626014 12200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.625692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:41763 (local address 127.0.0.1:56914)
0808 02:20:52.625838 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.625932 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:41763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 02:20:52.626312 12125 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.79.252:41763: connect: Connection refused (error 111)
I20260808 02:20:52.627496 12200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.627250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:41763 (local address 127.0.0.1:56920)
0808 02:20:52.627373 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.627422 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:41763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 02:20:52.627682 12196 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.252:41763: Network error: Client connection negotiation failed: client connection to 127.24.79.252:41763: connect: Connection refused (error 111)
I20260808 02:20:52.677017 12196 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } attempt: 1
I20260808 02:20:52.678184 12200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.677801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:41763 (local address 127.0.0.1:56928)
0808 02:20:52.677989 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.678073 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:41763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 02:20:52.679061 12200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.678763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:41763 (local address 127.0.0.1:56934)
0808 02:20:52.678873 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.678985 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:41763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 02:20:52.679272 12196 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.252:41763: Network error: Client connection negotiation failed: client connection to 127.24.79.252:41763: connect: Connection refused (error 111)
I20260808 02:20:52.715023 12083 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } attempt: 4
Aug 08 02:20:52 dist-test-slave-1vmg krb5kdc[11977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155652, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for kudu/127.24.79.253@KRBTEST.COM
I20260808 02:20:52.725790 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.715849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:40605 (local address 127.0.0.1:43712)
0808 02:20:52.716090 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.716108 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:52.716281 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:52.716584 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:52.716587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:52.716798 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:20:52.716929 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:52.716936 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.718138 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.718142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:52.719139 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:52.719153 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.719699 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.719704 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:52.719835 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:52.720356 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:20:52.720370 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:20:52.721943 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:20:52.724129 (+ 2186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:52.724133 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:52.724136 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:52.724378 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:52.724686 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:52.724690 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:52.724692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:52.724782 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:52.725217 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:20:52.725220 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:20:52.725332 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 02:20:52.725423 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 02:20:52.725593 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137}
I20260808 02:20:52.725790 12202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.715932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43712 (local address 127.24.79.253:40605)
0808 02:20:52.716094 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:52.716099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:52.716189 (+ 90us) server_negotiation.cc:407] Connection header received
0808 02:20:52.716402 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:52.716413 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:52.716470 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:52.716540 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:52.717079 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.717998 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.719297 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.719567 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.719669 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:52.722133 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:52.722136 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:52.722139 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:52.722174 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:52.723973 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.724493 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.724496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.724497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.724589 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.724870 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.724874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.724875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.725106 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:52.725209 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:52.725494 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 02:20:52.725622 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 02:20:52.727303 12083 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:20:52.728318 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.727860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:41763 (local address 127.0.0.1:56942)
0808 02:20:52.728056 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.728226 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:41763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260808 02:20:52.729048 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.728814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:41763 (local address 127.0.0.1:56944)
0808 02:20:52.728925 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.728979 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:41763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 02:20:52.729173 12083 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.252:41763: Network error: Client connection negotiation failed: client connection to 127.24.79.252:41763: connect: Connection refused (error 111)
I20260808 02:20:52.747486 12196 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } attempt: 2
I20260808 02:20:52.748509 12200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.748132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:41763 (local address 127.0.0.1:56960)
0808 02:20:52.748342 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.748415 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:41763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260808 02:20:52.749431 12200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.749150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:41763 (local address 127.0.0.1:56974)
0808 02:20:52.749303 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.749361 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:41763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260808 02:20:52.749601 12196 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.252:41763: Network error: Client connection negotiation failed: client connection to 127.24.79.252:41763: connect: Connection refused (error 111)
W20260808 02:20:52.755028 12203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:20:52.755371 12203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:20:52.755430 12203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:20:52.759683 12203 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:20:52.759805 12203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:20:52.759835 12203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:20:52.759859 12203 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:20:52.759889 12203 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:20:52.765872 12203 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:46207,127.24.79.253:40605,127.24.79.252:41763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.252.keytab
--principal=kudu/127.24.79.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.252:41763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.79.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:20:52.767241 12203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:20:52.768431 12203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:20:52.778582 12214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:52.778582 12210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:52.778582 12211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:52.778977 12203 server_base.cc:1034] running on GCE node
I20260808 02:20:52.779944 12203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:20:52.781469 12203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:20:52.782706 12203 hybrid_clock.cc:648] HybridClock initialized: now 1786155652782661 us; error 68 us; skew 500 ppm
Aug 08 02:20:52 dist-test-slave-1vmg krb5kdc[11977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155652, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:20:52.786454 12203 init.cc:377] Logged in from keytab as kudu/127.24.79.252@KRBTEST.COM (short username kudu)
I20260808 02:20:52.787674 12203 webserver.cc:460] Webserver started at http://127.24.79.252:42027/ using document root <none> and password file <none>
I20260808 02:20:52.788336 12203 fs_manager.cc:359] Metadata directory not provided
I20260808 02:20:52.788327 12083 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } attempt: 1
I20260808 02:20:52.788420 12203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:20:52.788664 12203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:20:52.789175 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.788881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:41763 (local address 127.0.0.1:56976)
0808 02:20:52.789022 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.789090 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:41763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 02:20:52.789915 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.789672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:41763 (local address 127.0.0.1:56990)
0808 02:20:52.789794 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.789848 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:41763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 02:20:52.790045 12083 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.252:41763: Network error: Client connection negotiation failed: client connection to 127.24.79.252:41763: connect: Connection refused (error 111)
I20260808 02:20:52.791316 12203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-2/data/instance:
uuid: "4d440e4cab6b40e0b2d239fa3b781da6"
format_stamp: "Formatted at 2026-08-08 02:20:52 on dist-test-slave-1vmg"
server_key: "0a7be64f460d62f2ce4c26f77322fe23"
server_key_iv: "3ec89420f24ef3c0df60d0d2b8e56ab0"
server_key_version: "encryptionkey@0"
I20260808 02:20:52.791965 12203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-2/wal/instance:
uuid: "4d440e4cab6b40e0b2d239fa3b781da6"
format_stamp: "Formatted at 2026-08-08 02:20:52 on dist-test-slave-1vmg"
server_key: "0a7be64f460d62f2ce4c26f77322fe23"
server_key_iv: "3ec89420f24ef3c0df60d0d2b8e56ab0"
server_key_version: "encryptionkey@0"
I20260808 02:20:52.796412 12203 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260808 02:20:52.799229 12223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:52.800716 12203 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:20:52.800877 12203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-2/wal
uuid: "4d440e4cab6b40e0b2d239fa3b781da6"
format_stamp: "Formatted at 2026-08-08 02:20:52 on dist-test-slave-1vmg"
server_key: "0a7be64f460d62f2ce4c26f77322fe23"
server_key_iv: "3ec89420f24ef3c0df60d0d2b8e56ab0"
server_key_version: "encryptionkey@0"
I20260808 02:20:52.801002 12203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:20:52.836339 12203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:20:52.840559 12203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:20:52.840811 12203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:20:52.840831 12196 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } attempt: 3
I20260808 02:20:52.841785 12200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.841419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:41763 (local address 127.0.0.1:57002)
0808 02:20:52.841574 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.841647 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:41763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 02:20:52.842615 12200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.842339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:41763 (local address 127.0.0.1:57014)
0808 02:20:52.842496 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.842548 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:41763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260808 02:20:52.842808 12196 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.252:41763: Network error: Client connection negotiation failed: client connection to 127.24.79.252:41763: connect: Connection refused (error 111)
I20260808 02:20:52.849447 12203 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.252:41763
I20260808 02:20:52.849516 12297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.252:41763 every 8 connection(s)
I20260808 02:20:52.850615 12203 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-2/data/info.pb
I20260808 02:20:52.854539 12298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:20:52.855082 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 12203
I20260808 02:20:52.855336 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155464663577-24895-0/minicluster-data/master-2/wal/instance
I20260808 02:20:52.855667 24895 external_mini_cluster.cc:1428] Setting key 2051cc656c2748d8e4660cdd5908d409
I20260808 02:20:52.857220 12083 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } attempt: 2
I20260808 02:20:52.860730 12298 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46207 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:20:52 dist-test-slave-1vmg krb5kdc[11977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155652, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for kudu/127.24.79.252@KRBTEST.COM
Aug 08 02:20:52 dist-test-slave-1vmg krb5kdc[11977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.252@KRBTEST.COM
Aug 08 02:20:52 dist-test-slave-1vmg krb5kdc[11977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155652, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.252@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:20:52.872260 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.857739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:41763 (local address 127.0.0.1:57026)
0808 02:20:52.857878 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.857892 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:52.857968 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:52.860659 (+ 2691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:52.860662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:52.860826 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:20:52.861098 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:52.861108 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.862398 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.862403 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:52.863317 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:52.863335 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.864641 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.864645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:52.864756 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:52.865369 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:20:52.865385 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:20:52.866940 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:20:52.869945 (+ 3005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:52.869949 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:52.869951 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:52.870160 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:52.870619 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:52.870623 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:52.870625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:52.870744 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:52.871660 (+ 916us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:20:52.871664 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:20:52.871787 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 02:20:52.871888 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 02:20:52.872067 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 02:20:52.873709 12301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.857284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57024 (local address 127.24.79.252:41763)
0808 02:20:52.857859 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:52.857882 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:52.857932 (+ 50us) server_negotiation.cc:407] Connection header received
0808 02:20:52.858797 (+ 865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:52.858827 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:52.859330 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:52.859696 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:52.860635 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.862182 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.863486 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.864440 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.864705 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:52.867849 (+ 3144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:52.867853 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:52.867856 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:52.867882 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:52.869674 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.870280 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.870287 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.870295 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.870470 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.870863 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.870866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.870868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.871511 (+ 643us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:52.871733 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:52.873283 (+ 1550us) server_negotiation.cc:299] Negotiation successful
0808 02:20:52.873407 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":267,"server-negotiator.queue_time_us":386,"spinlock_wait_cycles":43904,"thread_start_us":159,"threads_started":1}
I20260808 02:20:52.873721 12304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.858179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57026 (local address 127.24.79.252:41763)
0808 02:20:52.858493 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:52.858501 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:52.858516 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:20:52.858808 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:52.858829 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:52.859328 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:52.859696 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:52.861257 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.862192 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.863563 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.864440 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.864696 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:52.867149 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:52.867157 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:52.867173 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:52.867211 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:52.869778 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.870280 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.870288 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.870295 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.870476 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.870888 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.870891 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.870892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.871508 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:52.871679 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:52.872101 (+ 422us) server_negotiation.cc:299] Negotiation successful
0808 02:20:52.872471 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":175,"server-negotiator.queue_time_us":240,"spinlock_wait_cycles":39296,"thread_start_us":133,"threads_started":1}
I20260808 02:20:52.877162 12086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.864745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54516 (local address 127.24.79.254:46207)
0808 02:20:52.864874 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:52.864878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:52.864892 (+ 14us) server_negotiation.cc:407] Connection header received
0808 02:20:52.864930 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:52.864932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:52.864976 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:52.865064 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:52.866054 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.866830 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.868520 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.869095 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.869221 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:52.872871 (+ 3650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:52.872874 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:52.872876 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:52.872900 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:52.874820 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.875469 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.875473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.875474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.875578 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.875914 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.875917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.875919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.876277 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:52.876379 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:52.876843 (+ 464us) server_negotiation.cc:299] Negotiation successful
0808 02:20:52.876989 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:20:52.877187 12306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.863805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:46207 (local address 127.0.0.1:54516)
0808 02:20:52.864252 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.864314 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:52.864449 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:52.865164 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:52.865171 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:52.865739 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:20:52.865895 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:52.865902 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.866990 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.866994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:52.867966 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:52.867980 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.869271 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.869276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:52.869427 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:52.870360 (+ 933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:20:52.870374 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:20:52.872724 (+ 2350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:20:52.875023 (+ 2299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:52.875035 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:52.875041 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:52.875326 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:52.875705 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:52.875708 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:52.875710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:52.875813 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:52.876440 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:20:52.876447 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:20:52.876595 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 02:20:52.876757 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 02:20:52.876997 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"spinlock_wait_cycles":768,"thread_start_us":121,"threads_started":1}
I20260808 02:20:52.878734 12298 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:20:52.879344 12083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bc42ebef0e2436ba840ed71ccb17f37: Bootstrap starting.
I20260808 02:20:52.882828 12083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bc42ebef0e2436ba840ed71ccb17f37: Neither blocks nor log segments found. Creating new log.
Aug 08 02:20:52 dist-test-slave-1vmg krb5kdc[11977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:20:52.883985 12083 log.cc:824] T 00000000000000000000000000000000 P 4bc42ebef0e2436ba840ed71ccb17f37: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:20:52 dist-test-slave-1vmg krb5kdc[11977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155652, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.252@KRBTEST.COM for kudu/127.24.79.253@KRBTEST.COM
I20260808 02:20:52.886638 12083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bc42ebef0e2436ba840ed71ccb17f37: No bootstrap required, opened a new log
I20260808 02:20:52.888469 12086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.877035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54518 (local address 127.24.79.254:46207)
0808 02:20:52.877419 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:52.877423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:52.877938 (+ 515us) server_negotiation.cc:407] Connection header received
0808 02:20:52.878094 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:52.878097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:52.878158 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:52.878289 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:52.879388 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.880101 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.880961 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.881188 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.881280 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:52.884126 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:52.884131 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:52.884134 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:52.884168 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:52.885804 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.886633 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.886637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.886639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.886741 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.887099 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.887103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.887105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.887356 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:52.887557 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:52.888131 (+ 574us) server_negotiation.cc:299] Negotiation successful
0808 02:20:52.888292 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":98,"threads_started":1}
I20260808 02:20:52.889232 12202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.879747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43718 (local address 127.24.79.253:40605)
0808 02:20:52.879892 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:52.879896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:52.879912 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:20:52.879956 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:52.879959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:52.880002 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:52.880083 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:52.880575 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.881327 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.882485 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.882805 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.882893 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:52.885396 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:52.885400 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:52.885403 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:52.885435 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:52.886787 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.887327 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.887330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.887332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.887423 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.887742 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.887745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.887747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.887933 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:52.888019 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:52.888959 (+ 940us) server_negotiation.cc:299] Negotiation successful
0808 02:20:52.889082 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 02:20:52.889228 12306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.879409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:40605 (local address 127.0.0.1:43718)
0808 02:20:52.879736 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.879751 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:52.879861 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:52.880120 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:52.880123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:52.880314 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:20:52.880423 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:52.880429 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.881489 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.881492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:52.882330 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:52.882344 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.883121 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.883126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:52.883223 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:52.883667 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:20:52.883681 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:20:52.885205 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:20:52.886905 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:52.886908 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:52.886909 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:52.887202 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:52.887541 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:52.887544 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:52.887546 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:52.887644 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:52.888647 (+ 1003us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:20:52.888650 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:20:52.888768 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 02:20:52.888878 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 02:20:52.889042 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247}
I20260808 02:20:52.890363 12298 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:20:52.890887 12083 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bc42ebef0e2436ba840ed71ccb17f37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc42ebef0e2436ba840ed71ccb17f37" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46207 } } peers { permanent_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } } peers { permanent_uuid: "4d440e4cab6b40e0b2d239fa3b781da6" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } }
I20260808 02:20:52.891229 12083 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4bc42ebef0e2436ba840ed71ccb17f37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:20:52.891292 12083 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bc42ebef0e2436ba840ed71ccb17f37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bc42ebef0e2436ba840ed71ccb17f37, State: Initialized, Role: FOLLOWER
I20260808 02:20:52.891817 12083 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bc42ebef0e2436ba840ed71ccb17f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc42ebef0e2436ba840ed71ccb17f37" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46207 } } peers { permanent_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } } peers { permanent_uuid: "4d440e4cab6b40e0b2d239fa3b781da6" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } }
I20260808 02:20:52.892395 12310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bc42ebef0e2436ba840ed71ccb17f37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc42ebef0e2436ba840ed71ccb17f37" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46207 } } peers { permanent_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } } peers { permanent_uuid: "4d440e4cab6b40e0b2d239fa3b781da6" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } } }
I20260808 02:20:52.892644 12310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bc42ebef0e2436ba840ed71ccb17f37 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:20:52.893411 12083 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bc42ebef0e2436ba840ed71ccb17f37 [sys.catalog]: configured and running, proceeding with master startup.
Aug 08 02:20:52 dist-test-slave-1vmg krb5kdc[11977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155652, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.252@KRBTEST.COM for kudu/127.24.79.252@KRBTEST.COM
Aug 08 02:20:52 dist-test-slave-1vmg krb5kdc[11977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.253@KRBTEST.COM
I20260808 02:20:52.901340 12306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.891110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:41763 (local address 127.0.0.1:57036)
0808 02:20:52.891266 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.891281 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:52.891371 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:52.891976 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:52.891979 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:52.892156 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:20:52.892252 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:52.892259 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.893321 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.893324 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:52.894278 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:52.894294 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.895099 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.895102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:52.895163 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:52.895762 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:20:52.895780 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:20:52.897393 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:20:52.899710 (+ 2317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:52.899713 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:52.899715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:52.899964 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:52.900238 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:52.900241 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:52.900243 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:52.900319 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:52.900829 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:20:52.900831 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:20:52.900921 (+ 90us) client_negotiation.cc:769] Sending connection context
0808 02:20:52.901011 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 02:20:52.901159 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
W20260808 02:20:52.901330 12326 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4bc42ebef0e2436ba840ed71ccb17f37: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:20:52.901420 12326 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:20:52.901798 12304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.891126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57036 (local address 127.24.79.252:41763)
0808 02:20:52.891707 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:52.891711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:52.891723 (+ 12us) server_negotiation.cc:407] Connection header received
0808 02:20:52.891755 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:52.891758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:52.891797 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:52.891870 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:52.892375 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.893189 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.894434 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.894809 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.894889 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:52.897547 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:52.897550 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:52.897553 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:52.897578 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:52.899597 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.900062 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.900065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.900067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.900141 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.900396 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.900399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.900400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.900702 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:52.900797 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:52.901567 (+ 770us) server_negotiation.cc:299] Negotiation successful
0808 02:20:52.901695 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489}
I20260808 02:20:52.902699 12202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.890724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43730 (local address 127.24.79.253:40605)
0808 02:20:52.891352 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:52.891356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:52.891370 (+ 14us) server_negotiation.cc:407] Connection header received
0808 02:20:52.891410 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:52.891413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:52.891459 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:52.891527 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:52.892392 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.893196 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.894446 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.894815 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.894889 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:52.898522 (+ 3633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:52.898526 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:52.898529 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:52.898562 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:52.900231 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.900748 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.900752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.900754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.900844 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.901231 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.901234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.901236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.901489 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:52.901719 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:52.902406 (+ 687us) server_negotiation.cc:299] Negotiation successful
0808 02:20:52.902544 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547}
I20260808 02:20:52.899878 11993 master_runner.cc:186] Error getting master registration for 127.24.79.253:40605: OK, instance_id { permanent_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" instance_seqno: 1786155652574684 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 02:20:52.904143 11993 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:20:52.905099 12298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d440e4cab6b40e0b2d239fa3b781da6: Bootstrap starting.
I20260808 02:20:52.907893 12298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d440e4cab6b40e0b2d239fa3b781da6: Neither blocks nor log segments found. Creating new log.
I20260808 02:20:52.908869 12298 log.cc:824] T 00000000000000000000000000000000 P 4d440e4cab6b40e0b2d239fa3b781da6: Log is configured to *not* fsync() on all Append() calls
I20260808 02:20:52.911098 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:20:52.911867 12298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d440e4cab6b40e0b2d239fa3b781da6: No bootstrap required, opened a new log
I20260808 02:20:52.916074 12298 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d440e4cab6b40e0b2d239fa3b781da6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc42ebef0e2436ba840ed71ccb17f37" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46207 } } peers { permanent_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } } peers { permanent_uuid: "4d440e4cab6b40e0b2d239fa3b781da6" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } }
I20260808 02:20:52.916400 12298 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d440e4cab6b40e0b2d239fa3b781da6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:20:52.916460 12298 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d440e4cab6b40e0b2d239fa3b781da6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d440e4cab6b40e0b2d239fa3b781da6, State: Initialized, Role: FOLLOWER
I20260808 02:20:52.917034 12298 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d440e4cab6b40e0b2d239fa3b781da6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc42ebef0e2436ba840ed71ccb17f37" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46207 } } peers { permanent_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } } peers { permanent_uuid: "4d440e4cab6b40e0b2d239fa3b781da6" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } }
I20260808 02:20:52.918659 12298 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d440e4cab6b40e0b2d239fa3b781da6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:20:52.925249 12086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.914359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54532 (local address 127.24.79.254:46207)
0808 02:20:52.914485 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:52.914489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:52.914756 (+ 267us) server_negotiation.cc:407] Connection header received
0808 02:20:52.915034 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:52.915037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:52.915096 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:52.915150 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:52.916172 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.916925 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.918336 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.918651 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.918770 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:52.920766 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:52.920770 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:52.920773 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:52.920803 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:52.922431 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.922959 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.922963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.922964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.923060 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.924161 (+ 1101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.924165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.924167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.924434 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:52.924586 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:52.924939 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 02:20:52.925064 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:20:52.919471 12340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d440e4cab6b40e0b2d239fa3b781da6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc42ebef0e2436ba840ed71ccb17f37" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46207 } } peers { permanent_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } } peers { permanent_uuid: "4d440e4cab6b40e0b2d239fa3b781da6" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } } }
I20260808 02:20:52.928030 12340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d440e4cab6b40e0b2d239fa3b781da6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:20:52.928483 12304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.914493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57046 (local address 127.24.79.252:41763)
0808 02:20:52.914604 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:52.914620 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:52.917690 (+ 3070us) server_negotiation.cc:407] Connection header received
0808 02:20:52.917858 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:52.917861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:52.917909 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:52.917955 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:52.918889 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.920035 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.921542 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.922154 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.922241 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:52.923906 (+ 1665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:52.923909 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:52.923913 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:52.923942 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:52.926054 (+ 2112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.926595 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.926598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.926600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.926701 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.927061 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.927064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.927066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.927502 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:52.927654 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:52.928176 (+ 522us) server_negotiation.cc:299] Negotiation successful
0808 02:20:52.928323 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 02:20:52.926434 12203 master_runner.cc:186] Error getting master registration for 127.24.79.253:40605: OK, instance_id { permanent_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" instance_seqno: 1786155652574684 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 02:20:52.931744 12203 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 02:20:52.933563 12356 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4d440e4cab6b40e0b2d239fa3b781da6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:20:52.933736 12356 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:20:52.936128 12202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.913775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43746 (local address 127.24.79.253:40605)
0808 02:20:52.913910 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:52.913915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:52.915129 (+ 1214us) server_negotiation.cc:407] Connection header received
0808 02:20:52.915320 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:52.915323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:52.915374 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:52.915423 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:52.916366 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.917144 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.929068 (+ 11924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.929438 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.929558 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:52.931503 (+ 1945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:52.931507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:52.931510 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:52.931543 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:52.933495 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.934176 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.934180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.934181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.934371 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.935103 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.935108 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.935110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.935364 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:52.935544 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:52.935819 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 02:20:52.935958 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 02:20:52.986002 12196 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } attempt: 4
Aug 08 02:20:52 dist-test-slave-1vmg krb5kdc[11977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155652, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.253@KRBTEST.COM for kudu/127.24.79.252@KRBTEST.COM
I20260808 02:20:52.997714 12304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.986828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57056 (local address 127.24.79.252:41763)
0808 02:20:52.986997 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:52.987002 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:52.987089 (+ 87us) server_negotiation.cc:407] Connection header received
0808 02:20:52.987249 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:52.987253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:52.987315 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:52.987437 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:52.987988 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.988783 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.989963 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.990307 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.990403 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:52.993408 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:52.993413 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:52.993416 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:52.993451 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:52.995352 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.996163 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.996168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.996170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.996296 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:52.996665 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:52.996669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:52.996671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:52.996948 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:52.997062 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:52.997390 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 02:20:52.997544 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:20:52.997711 12200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:52.986721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:41763 (local address 127.0.0.1:57056)
0808 02:20:52.986996 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:52.987014 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:52.987131 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:52.987486 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:52.987489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:52.987737 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:20:52.987869 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:52.987877 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.988941 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.988945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:52.989806 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:52.989819 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:52.990553 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:52.990557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:52.990647 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:52.991312 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:20:52.991328 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:20:52.993191 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:20:52.995589 (+ 2398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:52.995594 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:52.995597 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:52.995981 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:52.996406 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:52.996409 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:52.996412 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:52.996523 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:52.997059 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:20:52.997062 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:20:52.997202 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 02:20:52.997324 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 02:20:52.997511 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260808 02:20:53.002230 12196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1: Bootstrap starting.
I20260808 02:20:53.006166 12196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1: Neither blocks nor log segments found. Creating new log.
I20260808 02:20:53.007350 12196 log.cc:824] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1: Log is configured to *not* fsync() on all Append() calls
I20260808 02:20:53.010432 12196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1: No bootstrap required, opened a new log
I20260808 02:20:53.015558 12196 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc42ebef0e2436ba840ed71ccb17f37" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46207 } } peers { permanent_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } } peers { permanent_uuid: "4d440e4cab6b40e0b2d239fa3b781da6" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } }
I20260808 02:20:53.015887 12196 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:20:53.015951 12196 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8b1a60651d455f9e68f3c4dfb2abb1, State: Initialized, Role: FOLLOWER
I20260808 02:20:53.016496 12196 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc42ebef0e2436ba840ed71ccb17f37" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46207 } } peers { permanent_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } } peers { permanent_uuid: "4d440e4cab6b40e0b2d239fa3b781da6" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } }
I20260808 02:20:53.017514 12360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc42ebef0e2436ba840ed71ccb17f37" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46207 } } peers { permanent_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } } peers { permanent_uuid: "4d440e4cab6b40e0b2d239fa3b781da6" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } } }
I20260808 02:20:53.017889 12360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:20:53.018293 12196 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 02:20:53.024837 12372 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:20:53.025066 12372 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:20:53.030779 12091 master_runner.cc:263] No leader master found from master 2c8b1a60651d455f9e68f3c4dfb2abb1
I20260808 02:20:53.030949 12091 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:20:53.038269 12360 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:20:53.038555 12360 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc42ebef0e2436ba840ed71ccb17f37" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46207 } } peers { permanent_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } } peers { permanent_uuid: "4d440e4cab6b40e0b2d239fa3b781da6" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } }
I20260808 02:20:53.040130 12360 leader_election.cc:288] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bc42ebef0e2436ba840ed71ccb17f37 (127.24.79.254:46207), 4d440e4cab6b40e0b2d239fa3b781da6 (127.24.79.252:41763)
I20260808 02:20:53.041143 12258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d440e4cab6b40e0b2d239fa3b781da6" is_pre_election: true
I20260808 02:20:53.041589 12258 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4d440e4cab6b40e0b2d239fa3b781da6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8b1a60651d455f9e68f3c4dfb2abb1 in term 0.
I20260808 02:20:53.045251 12051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bc42ebef0e2436ba840ed71ccb17f37" is_pre_election: true
I20260808 02:20:53.045735 12051 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4bc42ebef0e2436ba840ed71ccb17f37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c8b1a60651d455f9e68f3c4dfb2abb1 in term 0.
I20260808 02:20:53.046880 12125 leader_election.cc:302] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8b1a60651d455f9e68f3c4dfb2abb1, 4d440e4cab6b40e0b2d239fa3b781da6; no voters:
I20260808 02:20:53.048570 12360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:20:53.048694 12360 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:20:53.048765 12360 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:20:53.050416 12360 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc42ebef0e2436ba840ed71ccb17f37" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46207 } } peers { permanent_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } } peers { permanent_uuid: "4d440e4cab6b40e0b2d239fa3b781da6" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } }
I20260808 02:20:53.051432 12051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bc42ebef0e2436ba840ed71ccb17f37"
I20260808 02:20:53.051606 12051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bc42ebef0e2436ba840ed71ccb17f37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:20:53.053164 12051 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4bc42ebef0e2436ba840ed71ccb17f37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8b1a60651d455f9e68f3c4dfb2abb1 in term 1.
I20260808 02:20:53.054138 12258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d440e4cab6b40e0b2d239fa3b781da6"
I20260808 02:20:53.054325 12258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d440e4cab6b40e0b2d239fa3b781da6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:20:53.055939 12258 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4d440e4cab6b40e0b2d239fa3b781da6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c8b1a60651d455f9e68f3c4dfb2abb1 in term 1.
I20260808 02:20:53.050935 12360 leader_election.cc:288] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [CANDIDATE]: Term 1 election: Requested vote from peers 4bc42ebef0e2436ba840ed71ccb17f37 (127.24.79.254:46207), 4d440e4cab6b40e0b2d239fa3b781da6 (127.24.79.252:41763)
I20260808 02:20:53.053650 12120 leader_election.cc:302] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c8b1a60651d455f9e68f3c4dfb2abb1, 4bc42ebef0e2436ba840ed71ccb17f37; no voters:
I20260808 02:20:53.056991 12360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:20:53.057231 12360 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8b1a60651d455f9e68f3c4dfb2abb1, State: Running, Role: LEADER
I20260808 02:20:53.057674 12360 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc42ebef0e2436ba840ed71ccb17f37" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46207 } } peers { permanent_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } } peers { permanent_uuid: "4d440e4cab6b40e0b2d239fa3b781da6" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } }
I20260808 02:20:53.060063 12360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c8b1a60651d455f9e68f3c4dfb2abb1. Latest consensus state: current_term: 1 leader_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc42ebef0e2436ba840ed71ccb17f37" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46207 } } peers { permanent_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } } peers { permanent_uuid: "4d440e4cab6b40e0b2d239fa3b781da6" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } } }
I20260808 02:20:53.060228 12360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [sys.catalog]: This master's current role is: LEADER
I20260808 02:20:53.060686 12377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:20:53.061848 12377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260808 02:20:53.066579 12026 tablet.cc:2369] T 00000000000000000000000000000000 P 4bc42ebef0e2436ba840ed71ccb17f37: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:20:53.070982 12128 tablet.cc:2369] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:20:53.073463 12258 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4d440e4cab6b40e0b2d239fa3b781da6 [term 1 FOLLOWER]: Refusing update from remote peer 2c8b1a60651d455f9e68f3c4dfb2abb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:20:53.074484 12375 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d440e4cab6b40e0b2d239fa3b781da6" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:20:53.083613 12340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d440e4cab6b40e0b2d239fa3b781da6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c8b1a60651d455f9e68f3c4dfb2abb1. Latest consensus state: current_term: 1 leader_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc42ebef0e2436ba840ed71ccb17f37" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46207 } } peers { permanent_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } } peers { permanent_uuid: "4d440e4cab6b40e0b2d239fa3b781da6" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } } }
I20260808 02:20:53.083851 12340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d440e4cab6b40e0b2d239fa3b781da6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:20:53.086328 12051 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4bc42ebef0e2436ba840ed71ccb17f37 [term 1 FOLLOWER]: Refusing update from remote peer 2c8b1a60651d455f9e68f3c4dfb2abb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:20:53.087145 12375 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bc42ebef0e2436ba840ed71ccb17f37" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:20:53.092272 12310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bc42ebef0e2436ba840ed71ccb17f37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c8b1a60651d455f9e68f3c4dfb2abb1. Latest consensus state: current_term: 1 leader_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc42ebef0e2436ba840ed71ccb17f37" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46207 } } peers { permanent_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } } peers { permanent_uuid: "4d440e4cab6b40e0b2d239fa3b781da6" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } } }
I20260808 02:20:53.092456 12310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bc42ebef0e2436ba840ed71ccb17f37 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 02:20:53.092618 12230 tablet.cc:2369] T 00000000000000000000000000000000 P 4d440e4cab6b40e0b2d239fa3b781da6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:20:53.098572 12360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc42ebef0e2436ba840ed71ccb17f37" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46207 } } peers { permanent_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } } peers { permanent_uuid: "4d440e4cab6b40e0b2d239fa3b781da6" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } } }
I20260808 02:20:53.098765 12360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [sys.catalog]: This master's current role is: LEADER
I20260808 02:20:53.101351 12310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bc42ebef0e2436ba840ed71ccb17f37 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc42ebef0e2436ba840ed71ccb17f37" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46207 } } peers { permanent_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } } peers { permanent_uuid: "4d440e4cab6b40e0b2d239fa3b781da6" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } } }
I20260808 02:20:53.101511 12310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bc42ebef0e2436ba840ed71ccb17f37 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:20:53.101516 12377 catalog_manager.cc:1347] Generated new cluster ID: 241289f766c74bee8c936824163025ca
I20260808 02:20:53.101570 12377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:20:53.107933 12379 mvcc.cc:205] Tried to move back new op lower bound from 7316093554978103296 to 7316093554928611328. Current Snapshot: MvccSnapshot[applied={T|T < 7316093554978103296 or (T in {7316093554978103296})}]
I20260808 02:20:53.110795 12375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc42ebef0e2436ba840ed71ccb17f37" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46207 } } peers { permanent_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } } peers { permanent_uuid: "4d440e4cab6b40e0b2d239fa3b781da6" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } } }
I20260808 02:20:53.111002 12375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1 [sys.catalog]: This master's current role is: LEADER
I20260808 02:20:53.111750 12383 mvcc.cc:205] Tried to move back new op lower bound from 7316093554978103296 to 7316093554928611328. Current Snapshot: MvccSnapshot[applied={T|T < 7316093554978103296}]
I20260808 02:20:53.112496 12340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d440e4cab6b40e0b2d239fa3b781da6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bc42ebef0e2436ba840ed71ccb17f37" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46207 } } peers { permanent_uuid: "2c8b1a60651d455f9e68f3c4dfb2abb1" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 40605 } } peers { permanent_uuid: "4d440e4cab6b40e0b2d239fa3b781da6" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 41763 } } }
I20260808 02:20:53.112674 12340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d440e4cab6b40e0b2d239fa3b781da6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:20:53.113384 12380 mvcc.cc:205] Tried to move back new op lower bound from 7316093554978103296 to 7316093554928611328. Current Snapshot: MvccSnapshot[applied={T|T < 7316093554978103296}]
I20260808 02:20:53.117230 12377 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:20:53.118324 12377 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:20:53.131610 12377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c8b1a60651d455f9e68f3c4dfb2abb1: Generated new TSK 0
I20260808 02:20:53.132655 12377 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:20:53.903416 12326 catalog_manager.cc:1259] Loaded cluster ID: 241289f766c74bee8c936824163025ca
I20260808 02:20:53.903509 12326 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4bc42ebef0e2436ba840ed71ccb17f37: loading cluster ID for follower catalog manager: success
I20260808 02:20:53.905385 12326 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4bc42ebef0e2436ba840ed71ccb17f37: acquiring CA information for follower catalog manager: success
I20260808 02:20:53.906903 12326 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4bc42ebef0e2436ba840ed71ccb17f37: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:20:53.935823 12356 catalog_manager.cc:1259] Loaded cluster ID: 241289f766c74bee8c936824163025ca
I20260808 02:20:53.935935 12356 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4d440e4cab6b40e0b2d239fa3b781da6: loading cluster ID for follower catalog manager: success
I20260808 02:20:53.937799 12356 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4d440e4cab6b40e0b2d239fa3b781da6: acquiring CA information for follower catalog manager: success
I20260808 02:20:53.939224 12356 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4d440e4cab6b40e0b2d239fa3b781da6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:20:54.214743 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 11993
I20260808 02:20:54.230738 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 12091
I20260808 02:20:54.248541 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 12203
2026-08-08T02:20:54Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3454 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:20:54 dist-test-slave-1vmg krb5kdc[12396](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:20:54 dist-test-slave-1vmg krb5kdc[12396](info): set up 2 sockets
Aug 08 02:20:54 dist-test-slave-1vmg krb5kdc[12396](info): commencing operation
krb5kdc: starting...
W20260808 02:20:55.443830 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:20:55 dist-test-slave-1vmg krb5kdc[12396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:20:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:20:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:20:55.605015 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:38681
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:38681
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:20:55.733731 12416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:20:55.734083 12416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:20:55.734138 12416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:20:55.738544 12416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:20:55.738663 12416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:20:55.738693 12416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:20:55.738715 12416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:20:55.738730 12416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:20:55.745309 12416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:38681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:38681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:20:55.746626 12416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:20:55.747881 12416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:20:55.757709 12424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:55.758279 12427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:55.758296 12423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:55.758329 12416 server_base.cc:1034] running on GCE node
I20260808 02:20:55.759167 12416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:20:55.760546 12416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:20:55.761822 12416 hybrid_clock.cc:648] HybridClock initialized: now 1786155655761782 us; error 61 us; skew 500 ppm
Aug 08 02:20:55 dist-test-slave-1vmg krb5kdc[12396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155655, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:20:55.765218 12416 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:20:55.766396 12416 webserver.cc:460] Webserver started at http://127.24.79.254:39655/ using document root <none> and password file <none>
I20260808 02:20:55.767097 12416 fs_manager.cc:359] Metadata directory not provided
I20260808 02:20:55.767189 12416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:20:55.767588 12416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:20:55.770234 12416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "2234b52b84774a5184ce4927c1fdf3f5"
format_stamp: "Formatted at 2026-08-08 02:20:55 on dist-test-slave-1vmg"
server_key: "fd4463b06c54eaff6564fdc31532aea9"
server_key_iv: "2ccf63842ded0c84b867a8e69a1d214b"
server_key_version: "encryptionkey@0"
I20260808 02:20:55.771116 12416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "2234b52b84774a5184ce4927c1fdf3f5"
format_stamp: "Formatted at 2026-08-08 02:20:55 on dist-test-slave-1vmg"
server_key: "fd4463b06c54eaff6564fdc31532aea9"
server_key_iv: "2ccf63842ded0c84b867a8e69a1d214b"
server_key_version: "encryptionkey@0"
I20260808 02:20:55.776183 12416 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260808 02:20:55.779520 12435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:55.781258 12416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:20:55.781472 12416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "2234b52b84774a5184ce4927c1fdf3f5"
format_stamp: "Formatted at 2026-08-08 02:20:55 on dist-test-slave-1vmg"
server_key: "fd4463b06c54eaff6564fdc31532aea9"
server_key_iv: "2ccf63842ded0c84b867a8e69a1d214b"
server_key_version: "encryptionkey@0"
I20260808 02:20:55.781623 12416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:20:55.798193 12416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:20:55.801924 12416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:20:55.802138 12416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:20:55.810307 12416 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:38681
I20260808 02:20:55.810398 12509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:38681 every 8 connection(s)
I20260808 02:20:55.811563 12416 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:20:55.814651 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 12416
I20260808 02:20:55.814782 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:20:55.815014 12511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:20:55.815135 24895 external_mini_cluster.cc:1428] Setting key d76e499a467ec0d54f4ed7e93f188483
I20260808 02:20:55.823594 12511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2234b52b84774a5184ce4927c1fdf3f5: Bootstrap starting.
Aug 08 02:20:55 dist-test-slave-1vmg krb5kdc[12396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:20:55.826649 12511 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2234b52b84774a5184ce4927c1fdf3f5: Neither blocks nor log segments found. Creating new log.
I20260808 02:20:55.827714 12511 log.cc:824] T 00000000000000000000000000000000 P 2234b52b84774a5184ce4927c1fdf3f5: Log is configured to *not* fsync() on all Append() calls
I20260808 02:20:55.830579 12511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2234b52b84774a5184ce4927c1fdf3f5: No bootstrap required, opened a new log
I20260808 02:20:55.833012 12514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:55.816752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43728 (local address 127.24.79.254:38681)
0808 02:20:55.817276 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:55.817298 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:55.817347 (+ 49us) server_negotiation.cc:407] Connection header received
0808 02:20:55.818250 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:55.818283 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:55.818797 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:55.819339 (+ 542us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:55.820173 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:55.821747 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:55.822790 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:55.823571 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:55.823773 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:55.826886 (+ 3113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:55.826894 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:55.826948 (+ 54us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:55.826995 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:55.829002 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:55.829611 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:55.829617 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:55.829625 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:55.829803 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:55.830205 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:55.830209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:55.830211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:55.830846 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:55.831116 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:55.831468 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 02:20:55.831854 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":159,"threads_started":1}
I20260808 02:20:55.835781 12511 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2234b52b84774a5184ce4927c1fdf3f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2234b52b84774a5184ce4927c1fdf3f5" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38681 } }
I20260808 02:20:55.836130 12511 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2234b52b84774a5184ce4927c1fdf3f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:20:55.836293 12511 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2234b52b84774a5184ce4927c1fdf3f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2234b52b84774a5184ce4927c1fdf3f5, State: Initialized, Role: FOLLOWER
I20260808 02:20:55.837054 12511 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2234b52b84774a5184ce4927c1fdf3f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2234b52b84774a5184ce4927c1fdf3f5" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38681 } }
I20260808 02:20:55.837338 12511 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2234b52b84774a5184ce4927c1fdf3f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:20:55.837448 12511 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2234b52b84774a5184ce4927c1fdf3f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:20:55.837450 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:20:55.837576 12511 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2234b52b84774a5184ce4927c1fdf3f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:20:55.838778 12511 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2234b52b84774a5184ce4927c1fdf3f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2234b52b84774a5184ce4927c1fdf3f5" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38681 } }
I20260808 02:20:55.839223 12511 leader_election.cc:302] T 00000000000000000000000000000000 P 2234b52b84774a5184ce4927c1fdf3f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2234b52b84774a5184ce4927c1fdf3f5; no voters:
I20260808 02:20:55.839623 12511 leader_election.cc:288] T 00000000000000000000000000000000 P 2234b52b84774a5184ce4927c1fdf3f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:20:55.839756 12516 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2234b52b84774a5184ce4927c1fdf3f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:20:55.840008 12516 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2234b52b84774a5184ce4927c1fdf3f5 [term 1 LEADER]: Becoming Leader. State: Replica: 2234b52b84774a5184ce4927c1fdf3f5, State: Running, Role: LEADER
I20260808 02:20:55.840425 12516 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2234b52b84774a5184ce4927c1fdf3f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2234b52b84774a5184ce4927c1fdf3f5" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38681 } }
I20260808 02:20:55.841439 12511 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2234b52b84774a5184ce4927c1fdf3f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:20:55.842264 12517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2234b52b84774a5184ce4927c1fdf3f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2234b52b84774a5184ce4927c1fdf3f5. Latest consensus state: current_term: 1 leader_uuid: "2234b52b84774a5184ce4927c1fdf3f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2234b52b84774a5184ce4927c1fdf3f5" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38681 } } }
I20260808 02:20:55.842284 12519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2234b52b84774a5184ce4927c1fdf3f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2234b52b84774a5184ce4927c1fdf3f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2234b52b84774a5184ce4927c1fdf3f5" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38681 } } }
I20260808 02:20:55.842588 12519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2234b52b84774a5184ce4927c1fdf3f5 [sys.catalog]: This master's current role is: LEADER
I20260808 02:20:55.842900 12517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2234b52b84774a5184ce4927c1fdf3f5 [sys.catalog]: This master's current role is: LEADER
I20260808 02:20:55.843035 12533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:20:55.846369 12533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:20:55.853781 12533 catalog_manager.cc:1347] Generated new cluster ID: e6488708b3ac4ef1896eed9b8b5c66a0
I20260808 02:20:55.853901 12533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:20:55.870012 12533 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:20:55.871557 12533 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:20:55.881342 12533 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2234b52b84774a5184ce4927c1fdf3f5: Generated new TSK 0
I20260808 02:20:55.882402 12533 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:20:55.895560 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 12416
2026-08-08T02:20:55Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1644 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:20:55 dist-test-slave-1vmg krb5kdc[12549](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:20:55 dist-test-slave-1vmg krb5kdc[12549](info): set up 2 sockets
Aug 08 02:20:55 dist-test-slave-1vmg krb5kdc[12549](info): commencing operation
krb5kdc: starting...
W20260808 02:20:57.085821 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:20:57 dist-test-slave-1vmg krb5kdc[12549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:20:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:20:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:20:57.256038 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:43955
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:46371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:43955
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:20:57.387267 12565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:20:57.387617 12565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:20:57.387661 12565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:20:57.392062 12565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:20:57.392158 12565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:20:57.392179 12565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:20:57.392194 12565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:20:57.392210 12565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:20:57.398875 12565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:46371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:43955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:43955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:20:57.400336 12565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:20:57.401492 12565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:20:57.412220 12575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:57.412192 12572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:57.412352 12565 server_base.cc:1034] running on GCE node
W20260808 02:20:57.412192 12570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:57.413183 12565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:20:57.414559 12565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:20:57.415804 12565 hybrid_clock.cc:648] HybridClock initialized: now 1786155657415765 us; error 53 us; skew 500 ppm
Aug 08 02:20:57 dist-test-slave-1vmg krb5kdc[12549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155657, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:20:57.419523 12565 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:20:57.420718 12565 webserver.cc:460] Webserver started at http://127.24.79.254:42591/ using document root <none> and password file <none>
I20260808 02:20:57.421342 12565 fs_manager.cc:359] Metadata directory not provided
I20260808 02:20:57.421418 12565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:20:57.421624 12565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:20:57.424160 12565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "a26e629e3a0343b585265b3da367805b"
format_stamp: "Formatted at 2026-08-08 02:20:57 on dist-test-slave-1vmg"
server_key: "90e9ff9ac4d9d7d1d2bdaaa6af5aeee4"
server_key_iv: "abce51283da1efeeb6ae1b0e1a6b0174"
server_key_version: "encryptionkey@0"
I20260808 02:20:57.424804 12565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "a26e629e3a0343b585265b3da367805b"
format_stamp: "Formatted at 2026-08-08 02:20:57 on dist-test-slave-1vmg"
server_key: "90e9ff9ac4d9d7d1d2bdaaa6af5aeee4"
server_key_iv: "abce51283da1efeeb6ae1b0e1a6b0174"
server_key_version: "encryptionkey@0"
I20260808 02:20:57.429466 12565 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:20:57.432760 12590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:57.434409 12565 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:20:57.434585 12565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "a26e629e3a0343b585265b3da367805b"
format_stamp: "Formatted at 2026-08-08 02:20:57 on dist-test-slave-1vmg"
server_key: "90e9ff9ac4d9d7d1d2bdaaa6af5aeee4"
server_key_iv: "abce51283da1efeeb6ae1b0e1a6b0174"
server_key_version: "encryptionkey@0"
I20260808 02:20:57.434746 12565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:20:57.449090 12565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:20:57.452950 12565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:20:57.453187 12565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:20:57.461694 12565 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:43955
I20260808 02:20:57.461706 12664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:43955 every 8 connection(s)
I20260808 02:20:57.462882 12565 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:20:57.465838 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 12565
I20260808 02:20:57.465953 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:20:57.466255 24895 external_mini_cluster.cc:1428] Setting key bac3d5b0eef3fdfbf897808c8570c4ce
I20260808 02:20:57.466389 12665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:20:57.474956 12665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b: Bootstrap starting.
I20260808 02:20:57.477888 12665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b: Neither blocks nor log segments found. Creating new log.
Aug 08 02:20:57 dist-test-slave-1vmg krb5kdc[12549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:20:57.478817 12665 log.cc:824] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b: Log is configured to *not* fsync() on all Append() calls
I20260808 02:20:57.481801 12665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b: No bootstrap required, opened a new log
I20260808 02:20:57.485106 12668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:57.468263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45430 (local address 127.24.79.254:43955)
0808 02:20:57.468844 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:57.468865 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:57.469012 (+ 147us) server_negotiation.cc:407] Connection header received
0808 02:20:57.470308 (+ 1296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:57.470345 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:57.470897 (+ 552us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:57.471418 (+ 521us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:57.472316 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:57.473947 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:57.474977 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:57.475710 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:57.475876 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:57.478789 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:57.478797 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:57.478819 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:57.478865 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:57.481275 (+ 2410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:57.482063 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:57.482070 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:57.482084 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:57.482235 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:57.482694 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:57.482697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:57.482698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:57.483274 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:57.483495 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:57.483854 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 02:20:57.484277 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":155,"threads_started":1}
I20260808 02:20:57.486217 12665 raft_consensus.cc:348] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26e629e3a0343b585265b3da367805b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 43955 } }
I20260808 02:20:57.486570 12665 raft_consensus.cc:374] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:20:57.486630 12665 raft_consensus.cc:729] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a26e629e3a0343b585265b3da367805b, State: Initialized, Role: FOLLOWER
I20260808 02:20:57.487218 12665 consensus_queue.cc:260] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26e629e3a0343b585265b3da367805b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 43955 } }
I20260808 02:20:57.487423 12665 raft_consensus.cc:388] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:20:57.487498 12665 raft_consensus.cc:482] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:20:57.487603 12665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:20:57.488669 12665 raft_consensus.cc:504] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26e629e3a0343b585265b3da367805b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 43955 } }
I20260808 02:20:57.489070 12665 leader_election.cc:302] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a26e629e3a0343b585265b3da367805b; no voters:
I20260808 02:20:57.489482 12665 leader_election.cc:288] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:20:57.489647 12671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:20:57.489876 12671 raft_consensus.cc:686] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 1 LEADER]: Becoming Leader. State: Replica: a26e629e3a0343b585265b3da367805b, State: Running, Role: LEADER
I20260808 02:20:57.490307 12671 consensus_queue.cc:237] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26e629e3a0343b585265b3da367805b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 43955 } }
I20260808 02:20:57.490847 12665 sys_catalog.cc:564] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:20:57.491972 12674 sys_catalog.cc:455] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a26e629e3a0343b585265b3da367805b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26e629e3a0343b585265b3da367805b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 43955 } } }
I20260808 02:20:57.492123 12674 sys_catalog.cc:458] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [sys.catalog]: This master's current role is: LEADER
I20260808 02:20:57.492549 12679 sys_catalog.cc:455] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a26e629e3a0343b585265b3da367805b. Latest consensus state: current_term: 1 leader_uuid: "a26e629e3a0343b585265b3da367805b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26e629e3a0343b585265b3da367805b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 43955 } } }
I20260808 02:20:57.492658 12679 sys_catalog.cc:458] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [sys.catalog]: This master's current role is: LEADER
I20260808 02:20:57.492944 12687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:20:57.496206 12687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:20:57.502110 12687 catalog_manager.cc:1347] Generated new cluster ID: 3576271504874753978c9ac6f294582e
I20260808 02:20:57.502310 12687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:20:57.520229 12687 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:20:57.521351 12687 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:20:57.527599 12687 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b: Generated new TSK 0
I20260808 02:20:57.528648 12687 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.79.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
I20260808 02:20:57.576159 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:43955
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:46371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:20:57.708330 12703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:20:57.708647 12703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:20:57.708694 12703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:20:57.713074 12703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:20:57.713173 12703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:20:57.713238 12703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:20:57.719630 12703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:46371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:43955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:20:57.721011 12703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:20:57.722236 12703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:20:57.735165 12711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:57.735163 12713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:57.735172 12710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:57.735232 12703 server_base.cc:1034] running on GCE node
I20260808 02:20:57.735960 12703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:20:57.736785 12703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:20:57.737995 12703 hybrid_clock.cc:648] HybridClock initialized: now 1786155657737882 us; error 129 us; skew 500 ppm
Aug 08 02:20:57 dist-test-slave-1vmg krb5kdc[12549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155657, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:20:57.741638 12703 init.cc:377] Logged in from keytab as kudu/127.24.79.193@KRBTEST.COM (short username kudu)
I20260808 02:20:57.742822 12703 webserver.cc:460] Webserver started at http://127.24.79.193:34277/ using document root <none> and password file <none>
I20260808 02:20:57.743531 12703 fs_manager.cc:359] Metadata directory not provided
I20260808 02:20:57.743624 12703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:20:57.743885 12703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:20:57.746220 12703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "1b385bef71934d1e9c4e8db6120530fb"
format_stamp: "Formatted at 2026-08-08 02:20:57 on dist-test-slave-1vmg"
server_key: "0878b4f99b348c58f2ffffe5cd416dfd"
server_key_iv: "80d778bc23ee007807ea0b2a390cdc63"
server_key_version: "encryptionkey@0"
I20260808 02:20:57.746874 12703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "1b385bef71934d1e9c4e8db6120530fb"
format_stamp: "Formatted at 2026-08-08 02:20:57 on dist-test-slave-1vmg"
server_key: "0878b4f99b348c58f2ffffe5cd416dfd"
server_key_iv: "80d778bc23ee007807ea0b2a390cdc63"
server_key_version: "encryptionkey@0"
I20260808 02:20:57.751609 12703 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:20:57.754835 12723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:57.756436 12703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:20:57.756619 12703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "1b385bef71934d1e9c4e8db6120530fb"
format_stamp: "Formatted at 2026-08-08 02:20:57 on dist-test-slave-1vmg"
server_key: "0878b4f99b348c58f2ffffe5cd416dfd"
server_key_iv: "80d778bc23ee007807ea0b2a390cdc63"
server_key_version: "encryptionkey@0"
I20260808 02:20:57.756803 12703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:20:57.808301 12703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:20:57.812626 12703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:20:57.812883 12703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:20:57.813668 12703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:20:57.814738 12703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:20:57.814800 12703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:57.814862 12703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:20:57.814891 12703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:57.825472 12703 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:34449
I20260808 02:20:57.825558 12886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:34449 every 8 connection(s)
I20260808 02:20:57.826586 12703 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:20:57.836994 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 12703
I20260808 02:20:57.837164 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:20:57.837497 24895 external_mini_cluster.cc:1428] Setting key 22529ed3b11ea672d8d5d5cfe76b47d7
Aug 08 02:20:57 dist-test-slave-1vmg krb5kdc[12549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155657, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:20:57.847008 12668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:57.828944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:52061 (local address 127.24.79.254:43955)
0808 02:20:57.829127 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:57.829132 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:57.830361 (+ 1229us) server_negotiation.cc:407] Connection header received
0808 02:20:57.832252 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:57.832256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:57.832320 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:57.832406 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:57.834198 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:57.834781 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:57.836677 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:57.837055 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:57.837227 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:57.841487 (+ 4260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:57.841492 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:57.841495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:57.841532 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:57.843761 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:57.844664 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:57.844668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:57.844669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:57.844774 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:57.845349 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:57.845352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:57.845354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:57.845607 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:57.845705 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:57.846660 (+ 955us) server_negotiation.cc:299] Negotiation successful
0808 02:20:57.846819 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 02:20:57.848382 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:57.829446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.193:52061)
0808 02:20:57.830122 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:57.830187 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:57.831872 (+ 1685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:57.832721 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:57.832731 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:57.833180 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:20:57.833821 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:57.833840 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:57.835060 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:57.835064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:57.836260 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:57.836313 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:57.837490 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:57.837502 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:57.837761 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:57.838655 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:20:57.838680 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:20:57.841244 (+ 2564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:20:57.843968 (+ 2724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:57.843978 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:57.844003 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:57.844477 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:57.844890 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:57.844893 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:57.844895 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:57.845146 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:57.845756 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:20:57.845766 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:20:57.846255 (+ 489us) client_negotiation.cc:769] Sending connection context
0808 02:20:57.846531 (+ 276us) client_negotiation.cc:240] Negotiation successful
0808 02:20:57.847040 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":185,"threads_started":1}
I20260808 02:20:57.850210 12887 heartbeater.cc:347] Connected to a master server at 127.24.79.254:43955
I20260808 02:20:57.850703 12887 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.24.79.194@KRBTEST.COM; defaulting to no policy
I20260808 02:20:57.851454 12887 heartbeater.cc:511] Master 127.24.79.254:43955 requested a full tablet report, sending...
I20260808 02:20:57.853380 12615 ts_manager.cc:194] Registered new tserver with Master: 1b385bef71934d1e9c4e8db6120530fb (127.24.79.193:34449)
I20260808 02:20:57.854837 12615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.193@KRBTEST.COM'} at 127.24.79.193:52061
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
I20260808 02:20:57.881301 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:43955
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:46371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:20:58.015103 12892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:20:58.015455 12892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:20:58.015504 12892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:20:58.020313 12892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:20:58.020419 12892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:20:58.020485 12892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:20:58.026620 12892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:46371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:43955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:20:58.027961 12892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:20:58.029115 12892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:20:58.041638 12904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:58.041659 12901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:58.041625 12902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:58.042165 12892 server_base.cc:1034] running on GCE node
I20260808 02:20:58.042645 12892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:20:58.043424 12892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:20:58.044637 12892 hybrid_clock.cc:648] HybridClock initialized: now 1786155658044621 us; error 31 us; skew 500 ppm
Aug 08 02:20:58 dist-test-slave-1vmg krb5kdc[12549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155658, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:20:58.048081 12892 init.cc:377] Logged in from keytab as kudu/127.24.79.194@KRBTEST.COM (short username kudu)
I20260808 02:20:58.049419 12892 webserver.cc:460] Webserver started at http://127.24.79.194:35199/ using document root <none> and password file <none>
I20260808 02:20:58.050072 12892 fs_manager.cc:359] Metadata directory not provided
I20260808 02:20:58.050153 12892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:20:58.050395 12892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:20:58.052829 12892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "195a12ab501c46058d4d4733c3cf3835"
format_stamp: "Formatted at 2026-08-08 02:20:58 on dist-test-slave-1vmg"
server_key: "8235a3b1041433774ce3788405e123b0"
server_key_iv: "705a8737909dfee1ab0f9ef5cf8ad11d"
server_key_version: "encryptionkey@0"
I20260808 02:20:58.053674 12892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "195a12ab501c46058d4d4733c3cf3835"
format_stamp: "Formatted at 2026-08-08 02:20:58 on dist-test-slave-1vmg"
server_key: "8235a3b1041433774ce3788405e123b0"
server_key_iv: "705a8737909dfee1ab0f9ef5cf8ad11d"
server_key_version: "encryptionkey@0"
I20260808 02:20:58.058594 12892 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 02:20:58.061937 12914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:58.063745 12892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:20:58.063912 12892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "195a12ab501c46058d4d4733c3cf3835"
format_stamp: "Formatted at 2026-08-08 02:20:58 on dist-test-slave-1vmg"
server_key: "8235a3b1041433774ce3788405e123b0"
server_key_iv: "705a8737909dfee1ab0f9ef5cf8ad11d"
server_key_version: "encryptionkey@0"
I20260808 02:20:58.064067 12892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:20:58.077214 12892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:20:58.081508 12892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:20:58.081722 12892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:20:58.082651 12892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:20:58.084062 12892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:20:58.084188 12892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:58.084285 12892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:20:58.084319 12892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:58.094969 12892 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:34687
I20260808 02:20:58.094990 13066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:34687 every 8 connection(s)
I20260808 02:20:58.096312 12892 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:20:58.101444 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 12892
I20260808 02:20:58.101586 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:20:58.102007 24895 external_mini_cluster.cc:1428] Setting key a81f899b2e3e195d66c952ae2fcb099a
Aug 08 02:20:58 dist-test-slave-1vmg krb5kdc[12549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155658, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:20:58.113307 12668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:58.098751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:40779 (local address 127.24.79.254:43955)
0808 02:20:58.098901 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:58.098905 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:58.100043 (+ 1138us) server_negotiation.cc:407] Connection header received
0808 02:20:58.101603 (+ 1560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:58.101607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:58.101680 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:58.101803 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:58.103502 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.104206 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.105600 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.105910 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.106023 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:58.108906 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:58.108910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:58.108913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:58.108946 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:58.110622 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:58.111353 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:58.111357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:58.111359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:58.111471 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:58.111858 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:58.111861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:58.111863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:58.112139 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:58.112253 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:58.112973 (+ 720us) server_negotiation.cc:299] Negotiation successful
0808 02:20:58.113123 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
WARNING: no policy specified for kudu/127.24.79.195@KRBTEST.COM; defaulting to no policy
I20260808 02:20:58.114257 13071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:58.099196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.194:40779)
0808 02:20:58.099834 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:58.099893 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:58.101297 (+ 1404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:58.102060 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:58.102070 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:58.102581 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:20:58.103324 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:58.103343 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.104384 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.104387 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:58.105443 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:58.105456 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.106125 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.106128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:58.106320 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:58.107044 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:20:58.107064 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:20:58.108693 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:20:58.110803 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:58.110810 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:58.110826 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:58.111214 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:58.111578 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:58.111581 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:58.111582 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:58.111750 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:58.112338 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:20:58.112345 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:20:58.112661 (+ 316us) client_negotiation.cc:769] Sending connection context
0808 02:20:58.112855 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 02:20:58.113178 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":179,"threads_started":1}
I20260808 02:20:58.115852 13067 heartbeater.cc:347] Connected to a master server at 127.24.79.254:43955
I20260808 02:20:58.116151 13067 heartbeater.cc:464] Registering TS with master...
I20260808 02:20:58.116792 13067 heartbeater.cc:511] Master 127.24.79.254:43955 requested a full tablet report, sending...
I20260808 02:20:58.118320 12615 ts_manager.cc:194] Registered new tserver with Master: 195a12ab501c46058d4d4733c3cf3835 (127.24.79.194:34687)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.195@KRBTEST.COM" created.
I20260808 02:20:58.119100 12615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.194@KRBTEST.COM'} at 127.24.79.194:40779
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
I20260808 02:20:58.140542 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:43955
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:46371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:20:58.271256 13075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:20:58.271613 13075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:20:58.271664 13075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:20:58.276026 13075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:20:58.276124 13075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:20:58.276188 13075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:20:58.282212 13075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:46371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:43955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:20:58.283526 13075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:20:58.284668 13075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:20:58.295993 13087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:58.295993 13085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:58.296007 13084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:58.296376 13075 server_base.cc:1034] running on GCE node
I20260808 02:20:58.296934 13075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:20:58.297681 13075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:20:58.298892 13075 hybrid_clock.cc:648] HybridClock initialized: now 1786155658298866 us; error 40 us; skew 500 ppm
Aug 08 02:20:58 dist-test-slave-1vmg krb5kdc[12549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155658, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:20:58.302279 13075 init.cc:377] Logged in from keytab as kudu/127.24.79.195@KRBTEST.COM (short username kudu)
I20260808 02:20:58.303555 13075 webserver.cc:460] Webserver started at http://127.24.79.195:38049/ using document root <none> and password file <none>
I20260808 02:20:58.304344 13075 fs_manager.cc:359] Metadata directory not provided
I20260808 02:20:58.304441 13075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:20:58.304682 13075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:20:58.306995 13075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "204278d3b4cb446c932dba11b4dff15e"
format_stamp: "Formatted at 2026-08-08 02:20:58 on dist-test-slave-1vmg"
server_key: "176f6b683a4898042d84e3d4e91a9991"
server_key_iv: "6b63bf09a10b11ff9ceb5497ef5f5809"
server_key_version: "encryptionkey@0"
I20260808 02:20:58.307641 13075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "204278d3b4cb446c932dba11b4dff15e"
format_stamp: "Formatted at 2026-08-08 02:20:58 on dist-test-slave-1vmg"
server_key: "176f6b683a4898042d84e3d4e91a9991"
server_key_iv: "6b63bf09a10b11ff9ceb5497ef5f5809"
server_key_version: "encryptionkey@0"
I20260808 02:20:58.312444 13075 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 02:20:58.315644 13094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:58.317350 13075 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:20:58.317519 13075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "204278d3b4cb446c932dba11b4dff15e"
format_stamp: "Formatted at 2026-08-08 02:20:58 on dist-test-slave-1vmg"
server_key: "176f6b683a4898042d84e3d4e91a9991"
server_key_iv: "6b63bf09a10b11ff9ceb5497ef5f5809"
server_key_version: "encryptionkey@0"
I20260808 02:20:58.317655 13075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:20:58.334038 13075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:20:58.338184 13075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:20:58.338506 13075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:20:58.339222 13075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:20:58.340269 13075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:20:58.340327 13075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:58.340386 13075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:20:58.340421 13075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:58.350473 13075 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:41277
I20260808 02:20:58.350502 13258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:41277 every 8 connection(s)
I20260808 02:20:58.351778 13075 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:20:58.360368 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 13075
I20260808 02:20:58.360522 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:20:58.360874 24895 external_mini_cluster.cc:1428] Setting key 3d4541421062b22e07aec9fec330b3bb
Aug 08 02:20:58 dist-test-slave-1vmg krb5kdc[12549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155658, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:20:58.370298 12668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:58.354236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:38075 (local address 127.24.79.254:43955)
0808 02:20:58.354424 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:58.354429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:58.355596 (+ 1167us) server_negotiation.cc:407] Connection header received
0808 02:20:58.357246 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:58.357251 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:58.357325 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:58.357406 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:58.358999 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.359666 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.361563 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.361820 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.361940 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:58.365466 (+ 3526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:58.365470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:58.365473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:58.365505 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:58.367318 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:58.368260 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:58.368265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:58.368267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:58.368397 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:58.368895 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:58.368898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:58.368899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:58.369220 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:58.369325 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:58.369986 (+ 661us) server_negotiation.cc:299] Negotiation successful
0808 02:20:58.370121 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 02:20:58.371236 13263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:58.354669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.195:38075)
0808 02:20:58.355380 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:58.355437 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:58.356927 (+ 1490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:58.357681 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:58.357691 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:58.358171 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:20:58.358758 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:58.358775 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.359851 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.359854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:58.361389 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:58.361403 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.362139 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.362146 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:58.362368 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:58.363285 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:20:58.363306 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:20:58.365168 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:20:58.367512 (+ 2344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:58.367520 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:58.367565 (+ 45us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:58.368022 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:58.368551 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:58.368554 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:58.368556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:58.368760 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:58.369363 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:20:58.369370 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:20:58.369682 (+ 312us) client_negotiation.cc:769] Sending connection context
0808 02:20:58.369899 (+ 217us) client_negotiation.cc:240] Negotiation successful
0808 02:20:58.370238 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":227,"threads_started":1}
I20260808 02:20:58.372582 13259 heartbeater.cc:347] Connected to a master server at 127.24.79.254:43955
I20260808 02:20:58.372898 13259 heartbeater.cc:464] Registering TS with master...
I20260808 02:20:58.373503 13259 heartbeater.cc:511] Master 127.24.79.254:43955 requested a full tablet report, sending...
I20260808 02:20:58.374890 12609 ts_manager.cc:194] Registered new tserver with Master: 204278d3b4cb446c932dba11b4dff15e (127.24.79.195:41277)
I20260808 02:20:58.375592 12609 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.195@KRBTEST.COM'} at 127.24.79.195:38075
I20260808 02:20:58.375742 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:20:58.386651 12668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:58.378419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45434 (local address 127.24.79.254:43955)
0808 02:20:58.378574 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:58.378579 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:58.378755 (+ 176us) server_negotiation.cc:407] Connection header received
0808 02:20:58.378895 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:58.378898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:58.378992 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:58.379071 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:58.379867 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.380405 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.381283 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.381516 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.381613 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:58.383262 (+ 1649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:58.383266 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:58.383267 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:58.383300 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:58.384729 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:58.385258 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:58.385262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:58.385264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:58.385368 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:58.385675 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:58.385679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:58.385681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:58.385948 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:58.386091 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:58.386355 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 02:20:58.386483 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:20:58.390767 12609 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45434:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:20:58.394449 12609 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:20:58.411556 13278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:58.405048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:41277 (local address 127.0.0.1:37760)
0808 02:20:58.406085 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:58.406152 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:58.406318 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:58.407282 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:58.407285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:58.407308 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:20:58.407534 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:58.407542 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.408588 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.408592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:58.409908 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:58.409918 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.410992 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.410995 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:58.411085 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:58.411122 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 02:20:58.411278 (+ 156us) client_negotiation.cc:240] Negotiation successful
0808 02:20:58.411396 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":834,"mutex_wait_us":58,"spinlock_wait_cycles":8448,"thread_start_us":130,"threads_started":1}
I20260808 02:20:58.411626 13281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:58.405879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37760 (local address 127.24.79.195:41277)
0808 02:20:58.406658 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:58.406670 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:58.406693 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:20:58.406775 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:58.406781 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:58.407037 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:58.407452 (+ 415us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:20:58.407645 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.408427 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.410051 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.410824 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.410965 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:58.411139 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:58.411336 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 02:20:58.411451 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":167,"threads_started":1}
I20260808 02:20:58.411996 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:58.405309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:34687 (local address 127.0.0.1:58796)
0808 02:20:58.406105 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:58.406163 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:58.406319 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:58.407223 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:58.407229 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:58.407262 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:20:58.407478 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:58.407486 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.409058 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.409060 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:58.410424 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:58.410430 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.411543 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.411546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:58.411639 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:58.411711 (+ 72us) client_negotiation.cc:769] Sending connection context
0808 02:20:58.411791 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 02:20:58.411863 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"mutex_wait_us":127,"spinlock_wait_cycles":11136,"thread_start_us":117,"threads_started":1}
I20260808 02:20:58.412552 13282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:58.406003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58796 (local address 127.24.79.194:34687)
0808 02:20:58.406656 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:58.406671 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:58.406695 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:20:58.406776 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:58.406781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:58.407045 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:58.407654 (+ 609us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:20:58.407825 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.408926 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.410562 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.411407 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.411525 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:58.411670 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:58.412314 (+ 644us) server_negotiation.cc:299] Negotiation successful
0808 02:20:58.412426 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":140,"threads_started":1}
I20260808 02:20:58.413640 13276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:58.405436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:34449 (local address 127.0.0.1:53778)
0808 02:20:58.406327 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:58.406340 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:58.406412 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:58.407223 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:58.407229 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:58.407269 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:20:58.407487 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:58.407493 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.409014 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.409018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:58.410417 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:58.410430 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.413182 (+ 2752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.413186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:58.413287 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:58.413314 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 02:20:58.413389 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 02:20:58.413468 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":834,"mutex_wait_us":96,"spinlock_wait_cycles":20352,"thread_start_us":135,"threads_started":1}
I20260808 02:20:58.414120 13280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:58.405686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53778 (local address 127.24.79.193:34449)
0808 02:20:58.406658 (+ 972us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:58.406667 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:58.406693 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:20:58.406775 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:58.406781 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:58.407037 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:58.407207 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:20:58.407626 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.408863 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.411758 (+ 2895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.412577 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.412942 (+ 365us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:58.413116 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:58.413854 (+ 738us) server_negotiation.cc:299] Negotiation successful
0808 02:20:58.413972 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":876,"thread_start_us":138,"threads_started":1}
I20260808 02:20:58.415516 13168 tablet_service.cc:1467] Processing CreateTablet for tablet 55a2be20a9e34dd4a45e2e974b4f9bd1 (DEFAULT_TABLE table=test-table [id=de3b5c3d31c84ca1b13b5d0d67199e8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:20:58.416774 13168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55a2be20a9e34dd4a45e2e974b4f9bd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:20:58.416765 12793 tablet_service.cc:1467] Processing CreateTablet for tablet 55a2be20a9e34dd4a45e2e974b4f9bd1 (DEFAULT_TABLE table=test-table [id=de3b5c3d31c84ca1b13b5d0d67199e8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:20:58.417963 12793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55a2be20a9e34dd4a45e2e974b4f9bd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:20:58.418390 12980 tablet_service.cc:1467] Processing CreateTablet for tablet 55a2be20a9e34dd4a45e2e974b4f9bd1 (DEFAULT_TABLE table=test-table [id=de3b5c3d31c84ca1b13b5d0d67199e8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:20:58.419674 12980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55a2be20a9e34dd4a45e2e974b4f9bd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:20:58.425423 13283 tablet_bootstrap.cc:492] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb: Bootstrap starting.
I20260808 02:20:58.427045 13284 tablet_bootstrap.cc:492] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e: Bootstrap starting.
I20260808 02:20:58.427999 13283 tablet_bootstrap.cc:654] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb: Neither blocks nor log segments found. Creating new log.
I20260808 02:20:58.428284 13287 tablet_bootstrap.cc:492] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 195a12ab501c46058d4d4733c3cf3835: Bootstrap starting.
I20260808 02:20:58.430009 13284 tablet_bootstrap.cc:654] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e: Neither blocks nor log segments found. Creating new log.
I20260808 02:20:58.430065 13283 log.cc:824] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb: Log is configured to *not* fsync() on all Append() calls
I20260808 02:20:58.430799 13287 tablet_bootstrap.cc:654] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 195a12ab501c46058d4d4733c3cf3835: Neither blocks nor log segments found. Creating new log.
I20260808 02:20:58.430964 13284 log.cc:824] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:20:58.431716 13287 log.cc:824] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 195a12ab501c46058d4d4733c3cf3835: Log is configured to *not* fsync() on all Append() calls
I20260808 02:20:58.432687 13283 tablet_bootstrap.cc:492] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb: No bootstrap required, opened a new log
I20260808 02:20:58.432948 13283 ts_tablet_manager.cc:1397] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260808 02:20:58.432998 13284 tablet_bootstrap.cc:492] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e: No bootstrap required, opened a new log
I20260808 02:20:58.433188 13284 ts_tablet_manager.cc:1397] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 02:20:58.433862 13287 tablet_bootstrap.cc:492] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 195a12ab501c46058d4d4733c3cf3835: No bootstrap required, opened a new log
I20260808 02:20:58.434068 13287 ts_tablet_manager.cc:1397] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 195a12ab501c46058d4d4733c3cf3835: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 02:20:58.436786 13284 raft_consensus.cc:348] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:20:58.437130 13284 raft_consensus.cc:374] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:20:58.437228 13284 raft_consensus.cc:729] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 204278d3b4cb446c932dba11b4dff15e, State: Initialized, Role: FOLLOWER
I20260808 02:20:58.437748 13284 consensus_queue.cc:260] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:20:58.439247 13284 ts_tablet_manager.cc:1428] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 02:20:58.438895 13287 raft_consensus.cc:348] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 195a12ab501c46058d4d4733c3cf3835 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:20:58.439440 13287 raft_consensus.cc:374] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 195a12ab501c46058d4d4733c3cf3835 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:20:58.439522 13287 raft_consensus.cc:729] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 195a12ab501c46058d4d4733c3cf3835 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 195a12ab501c46058d4d4733c3cf3835, State: Initialized, Role: FOLLOWER
I20260808 02:20:58.438902 13283 raft_consensus.cc:348] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:20:58.439706 13283 raft_consensus.cc:374] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:20:58.439744 13259 heartbeater.cc:503] Master 127.24.79.254:43955 was elected leader, sending a full tablet report...
I20260808 02:20:58.439800 13283 raft_consensus.cc:729] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b385bef71934d1e9c4e8db6120530fb, State: Initialized, Role: FOLLOWER
I20260808 02:20:58.440249 13283 consensus_queue.cc:260] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:20:58.440223 13287 consensus_queue.cc:260] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 195a12ab501c46058d4d4733c3cf3835 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:20:58.441166 13067 heartbeater.cc:503] Master 127.24.79.254:43955 was elected leader, sending a full tablet report...
I20260808 02:20:58.441385 12887 heartbeater.cc:503] Master 127.24.79.254:43955 was elected leader, sending a full tablet report...
I20260808 02:20:58.441512 13287 ts_tablet_manager.cc:1428] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 195a12ab501c46058d4d4733c3cf3835: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260808 02:20:58.442579 13283 ts_tablet_manager.cc:1428] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260808 02:20:58.474195 13291 raft_consensus.cc:482] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:20:58.474499 13291 raft_consensus.cc:504] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:20:58.475797 13291 leader_election.cc:288] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 195a12ab501c46058d4d4733c3cf3835 (127.24.79.194:34687), 1b385bef71934d1e9c4e8db6120530fb (127.24.79.193:34449)
I20260808 02:20:58.481559 13282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:58.476431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:37161 (local address 127.24.79.194:34687)
0808 02:20:58.476568 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:58.476573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:58.476832 (+ 259us) server_negotiation.cc:407] Connection header received
0808 02:20:58.477011 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:58.477014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:58.477072 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:58.477171 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:20:58.477584 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.478184 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.479929 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.480619 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.480790 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:58.480856 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:58.481304 (+ 448us) server_negotiation.cc:299] Negotiation successful
0808 02:20:58.481412 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 02:20:58.481585 13280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:58.476293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:49651 (local address 127.24.79.193:34449)
0808 02:20:58.476432 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:58.476437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:58.476453 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:20:58.476632 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:58.476635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:58.476689 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:58.476758 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:20:58.477126 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.477847 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.479929 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.480607 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.480803 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:58.480856 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:58.481304 (+ 448us) server_negotiation.cc:299] Negotiation successful
0808 02:20:58.481453 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 02:20:58.481559 13263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:58.476131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:34449 (local address 127.24.79.195:49651)
0808 02:20:58.476330 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:58.476349 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:58.476489 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:58.476809 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:58.476812 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:58.476855 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:20:58.476987 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:58.477000 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.478078 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.478081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:58.479699 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:58.479714 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.480817 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.480821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:58.481018 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:58.481061 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 02:20:58.481230 (+ 169us) client_negotiation.cc:240] Negotiation successful
0808 02:20:58.481365 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":151,"spinlock_wait_cycles":6400}
I20260808 02:20:58.481572 13299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:58.476131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:34687 (local address 127.24.79.195:37161)
0808 02:20:58.476715 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:58.476734 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:58.476857 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:58.477311 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:58.477314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:58.477335 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:20:58.477449 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:58.477455 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.478403 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.478407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:58.479699 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:58.479715 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.480817 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.480821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:58.480989 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:58.481072 (+ 83us) client_negotiation.cc:769] Sending connection context
0808 02:20:58.481249 (+ 177us) client_negotiation.cc:240] Negotiation successful
0808 02:20:58.481365 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":164,"spinlock_wait_cycles":33280,"thread_start_us":155,"threads_started":1}
I20260808 02:20:58.482692 12822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55a2be20a9e34dd4a45e2e974b4f9bd1" candidate_uuid: "204278d3b4cb446c932dba11b4dff15e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b385bef71934d1e9c4e8db6120530fb" is_pre_election: true
I20260808 02:20:58.482808 13008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55a2be20a9e34dd4a45e2e974b4f9bd1" candidate_uuid: "204278d3b4cb446c932dba11b4dff15e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "195a12ab501c46058d4d4733c3cf3835" is_pre_election: true
I20260808 02:20:58.483105 12822 raft_consensus.cc:2454] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 204278d3b4cb446c932dba11b4dff15e in term 0.
I20260808 02:20:58.483299 13008 raft_consensus.cc:2454] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 195a12ab501c46058d4d4733c3cf3835 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 204278d3b4cb446c932dba11b4dff15e in term 0.
I20260808 02:20:58.483712 13097 leader_election.cc:302] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b385bef71934d1e9c4e8db6120530fb, 204278d3b4cb446c932dba11b4dff15e; no voters:
I20260808 02:20:58.484046 13291 raft_consensus.cc:2781] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:20:58.484144 13291 raft_consensus.cc:482] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:20:58.484179 13291 raft_consensus.cc:3037] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:20:58.485484 13291 raft_consensus.cc:504] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:20:58.485985 13291 leader_election.cc:288] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e [CANDIDATE]: Term 1 election: Requested vote from peers 195a12ab501c46058d4d4733c3cf3835 (127.24.79.194:34687), 1b385bef71934d1e9c4e8db6120530fb (127.24.79.193:34449)
I20260808 02:20:58.486465 13008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55a2be20a9e34dd4a45e2e974b4f9bd1" candidate_uuid: "204278d3b4cb446c932dba11b4dff15e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "195a12ab501c46058d4d4733c3cf3835"
I20260808 02:20:58.486531 12822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55a2be20a9e34dd4a45e2e974b4f9bd1" candidate_uuid: "204278d3b4cb446c932dba11b4dff15e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b385bef71934d1e9c4e8db6120530fb"
I20260808 02:20:58.486637 13008 raft_consensus.cc:3037] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 195a12ab501c46058d4d4733c3cf3835 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:20:58.486691 12822 raft_consensus.cc:3037] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:20:58.487908 13008 raft_consensus.cc:2454] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 195a12ab501c46058d4d4733c3cf3835 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 204278d3b4cb446c932dba11b4dff15e in term 1.
I20260808 02:20:58.487911 12822 raft_consensus.cc:2454] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 204278d3b4cb446c932dba11b4dff15e in term 1.
I20260808 02:20:58.488286 13097 leader_election.cc:302] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b385bef71934d1e9c4e8db6120530fb, 204278d3b4cb446c932dba11b4dff15e; no voters:
I20260808 02:20:58.488487 13291 raft_consensus.cc:2781] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:20:58.488788 13291 raft_consensus.cc:686] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e [term 1 LEADER]: Becoming Leader. State: Replica: 204278d3b4cb446c932dba11b4dff15e, State: Running, Role: LEADER
I20260808 02:20:58.489166 13291 consensus_queue.cc:237] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:20:58.493292 12615 catalog_manager.cc:5495] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e reported cstate change: term changed from 0 to 1, leader changed from <none> to 204278d3b4cb446c932dba11b4dff15e (127.24.79.195). New cstate: current_term: 1 leader_uuid: "204278d3b4cb446c932dba11b4dff15e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } health_report { overall_health: UNKNOWN } } }
I20260808 02:20:58.517804 13281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:58.513246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37774 (local address 127.24.79.195:41277)
0808 02:20:58.513405 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:58.513408 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:58.513425 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:20:58.513624 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:58.513628 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:58.513693 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:58.513796 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:20:58.514135 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.514687 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.515792 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:58.516032 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:58.516138 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:58.516626 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:20:58.517174 (+ 548us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:20:58.517418 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:58.517594 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 02:20:58.517672 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 02:20:58.526222 12822 raft_consensus.cc:1261] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Refusing update from remote peer 204278d3b4cb446c932dba11b4dff15e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:20:58.526222 13008 raft_consensus.cc:1261] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 195a12ab501c46058d4d4733c3cf3835 [term 1 FOLLOWER]: Refusing update from remote peer 204278d3b4cb446c932dba11b4dff15e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:20:58.527163 13300 consensus_queue.cc:1035] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:20:58.527477 13291 consensus_queue.cc:1035] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e [LEADER]: Connected to new peer: Peer: permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:20:58.537487 13309 mvcc.cc:205] Tried to move back new op lower bound from 7316093577312894976 to 7316093577174773760. Current Snapshot: MvccSnapshot[applied={T|T < 7316093577312894976}]
I20260808 02:20:58.548593 12614 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:20:58.548841 12614 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:20:58.552198 12614 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b: Sending DeleteTablet for 3 replicas of tablet 55a2be20a9e34dd4a45e2e974b4f9bd1
I20260808 02:20:58.553831 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 12703
I20260808 02:20:58.553828 12980 tablet_service.cc:1514] Processing DeleteTablet for tablet 55a2be20a9e34dd4a45e2e974b4f9bd1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:20:58 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:58796
I20260808 02:20:58.554019 13168 tablet_service.cc:1514] Processing DeleteTablet for tablet 55a2be20a9e34dd4a45e2e974b4f9bd1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:20:58 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:37760
I20260808 02:20:58.556763 13318 tablet_replica.cc:330] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e: stopping tablet replica
I20260808 02:20:58.557173 13318 raft_consensus.cc:2229] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:20:58.557586 13318 raft_consensus.cc:2258] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:20:58.559667 13318 ts_tablet_manager.cc:1905] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:20:58.563040 13318 ts_tablet_manager.cc:1918] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:20:58.563194 13318 log.cc:1196] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/55a2be20a9e34dd4a45e2e974b4f9bd1
I20260808 02:20:58.563657 13318 ts_tablet_manager.cc:1939] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 204278d3b4cb446c932dba11b4dff15e: Deleting consensus metadata
I20260808 02:20:58.565522 12592 catalog_manager.cc:4841] TS 204278d3b4cb446c932dba11b4dff15e (127.24.79.195:41277): tablet 55a2be20a9e34dd4a45e2e974b4f9bd1 (table test-table [id=de3b5c3d31c84ca1b13b5d0d67199e8b]) successfully deleted
I20260808 02:20:58.565965 13317 tablet_replica.cc:330] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 195a12ab501c46058d4d4733c3cf3835: stopping tablet replica
I20260808 02:20:58.566496 13317 raft_consensus.cc:2229] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 195a12ab501c46058d4d4733c3cf3835 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:20:58.567037 13317 raft_consensus.cc:2258] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 195a12ab501c46058d4d4733c3cf3835 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:20:58.573995 13317 ts_tablet_manager.cc:1905] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 195a12ab501c46058d4d4733c3cf3835: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:20:58.578830 13317 ts_tablet_manager.cc:1918] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 195a12ab501c46058d4d4733c3cf3835: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:20:58.579025 13317 log.cc:1196] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 195a12ab501c46058d4d4733c3cf3835: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/55a2be20a9e34dd4a45e2e974b4f9bd1
I20260808 02:20:58.579435 13317 ts_tablet_manager.cc:1939] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 195a12ab501c46058d4d4733c3cf3835: Deleting consensus metadata
W20260808 02:20:58.580984 12593 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.79.193:34449 (error 108)
I20260808 02:20:58.581272 12594 catalog_manager.cc:4841] TS 195a12ab501c46058d4d4733c3cf3835 (127.24.79.194:34687): tablet 55a2be20a9e34dd4a45e2e974b4f9bd1 (table test-table [id=de3b5c3d31c84ca1b13b5d0d67199e8b]) successfully deleted
I20260808 02:20:58.581342 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 12892
I20260808 02:20:58.586252 13276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:58.584292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:34449 (local address 127.0.0.1:53784)
0808 02:20:58.585989 (+ 1697us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:58.586113 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.193:34449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1566}
W20260808 02:20:58.587203 12593 catalog_manager.cc:4584] TS 1b385bef71934d1e9c4e8db6120530fb (127.24.79.193:34449): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 55a2be20a9e34dd4a45e2e974b4f9bd1: Network error: Client connection negotiation failed: client connection to 127.24.79.193:34449: connect: Connection refused (error 111)
I20260808 02:20:58.593901 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 13075
I20260808 02:20:58.613360 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 12565
I20260808 02:20:58.630275 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:43955
--webserver_interface=127.24.79.254
--webserver_port=42591
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:46371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:43955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:20:58.757103 13321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:20:58.757458 13321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:20:58.757524 13321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:20:58.761859 13321 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:20:58.761965 13321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:20:58.761994 13321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:20:58.762023 13321 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:20:58.762043 13321 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:20:58.768748 13321 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:46371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:43955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:43955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=42591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:20:58.770113 13321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:20:58.771322 13321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:20:58.789438 13328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:58.789492 13333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:58.789479 13330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:58.790319 13321 server_base.cc:1034] running on GCE node
I20260808 02:20:58.791306 13321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:20:58.792797 13321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:20:58.794031 13321 hybrid_clock.cc:648] HybridClock initialized: now 1786155658794004 us; error 49 us; skew 500 ppm
Aug 08 02:20:58 dist-test-slave-1vmg krb5kdc[12549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155658, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:20:58.797710 13321 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:20:58.799067 13321 webserver.cc:460] Webserver started at http://127.24.79.254:42591/ using document root <none> and password file <none>
I20260808 02:20:58.799790 13321 fs_manager.cc:359] Metadata directory not provided
I20260808 02:20:58.799902 13321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:20:58.804937 13321 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 02:20:58.807996 13343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:58.809571 13321 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:20:58.809762 13321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "a26e629e3a0343b585265b3da367805b"
format_stamp: "Formatted at 2026-08-08 02:20:57 on dist-test-slave-1vmg"
server_key: "90e9ff9ac4d9d7d1d2bdaaa6af5aeee4"
server_key_iv: "abce51283da1efeeb6ae1b0e1a6b0174"
server_key_version: "encryptionkey@0"
I20260808 02:20:58.810523 13321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:20:58.831605 13321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:20:58.835886 13321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:20:58.836117 13321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:20:58.844599 13321 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:43955
I20260808 02:20:58.844884 13419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:43955 every 8 connection(s)
I20260808 02:20:58.845942 13321 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:20:58.850270 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 13321
I20260808 02:20:58.850749 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:34449
--local_ip_for_outbound_sockets=127.24.79.193
--tserver_master_addrs=127.24.79.254:43955
--webserver_port=34277
--webserver_interface=127.24.79.193
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:46371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:20:58.851467 13420 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:20:58.853523 13420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b: Bootstrap starting.
I20260808 02:20:58.873399 13420 log.cc:824] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b: Log is configured to *not* fsync() on all Append() calls
I20260808 02:20:58.883970 13420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:20:58.884423 13420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b: Bootstrap complete.
I20260808 02:20:58.888855 13420 raft_consensus.cc:348] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26e629e3a0343b585265b3da367805b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 43955 } }
I20260808 02:20:58.889346 13420 raft_consensus.cc:729] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a26e629e3a0343b585265b3da367805b, State: Initialized, Role: FOLLOWER
I20260808 02:20:58.889932 13420 consensus_queue.cc:260] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26e629e3a0343b585265b3da367805b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 43955 } }
I20260808 02:20:58.890077 13420 raft_consensus.cc:388] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:20:58.890153 13420 raft_consensus.cc:482] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:20:58.890285 13420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:20:58.892716 13420 raft_consensus.cc:504] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26e629e3a0343b585265b3da367805b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 43955 } }
I20260808 02:20:58.893154 13420 leader_election.cc:302] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a26e629e3a0343b585265b3da367805b; no voters:
I20260808 02:20:58.893543 13420 leader_election.cc:288] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 02:20:58.893724 13425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 2 FOLLOWER]: Leader election won for term 2
I20260808 02:20:58.893968 13425 raft_consensus.cc:686] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 2 LEADER]: Becoming Leader. State: Replica: a26e629e3a0343b585265b3da367805b, State: Running, Role: LEADER
I20260808 02:20:58.894349 13425 consensus_queue.cc:237] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26e629e3a0343b585265b3da367805b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 43955 } }
I20260808 02:20:58.895076 13420 sys_catalog.cc:564] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:20:58.896063 13428 sys_catalog.cc:455] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a26e629e3a0343b585265b3da367805b. Latest consensus state: current_term: 2 leader_uuid: "a26e629e3a0343b585265b3da367805b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26e629e3a0343b585265b3da367805b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 43955 } } }
I20260808 02:20:58.896102 13426 sys_catalog.cc:455] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a26e629e3a0343b585265b3da367805b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26e629e3a0343b585265b3da367805b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 43955 } } }
I20260808 02:20:58.896229 13426 sys_catalog.cc:458] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [sys.catalog]: This master's current role is: LEADER
I20260808 02:20:58.896235 13428 sys_catalog.cc:458] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [sys.catalog]: This master's current role is: LEADER
I20260808 02:20:58.900717 13444 catalog_manager.cc:1259] Loaded cluster ID: 3576271504874753978c9ac6f294582e
I20260808 02:20:58.900805 13444 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b: loading cluster ID for follower catalog manager: success
W20260808 02:20:58.901770 13444 catalog_manager.cc:1576] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260808 02:20:58.901852 13444 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260808 02:20:58.901940 13437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:20:58.904266 13437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:20:58.904778 13437 catalog_manager.cc:1259] Loaded cluster ID: 3576271504874753978c9ac6f294582e
I20260808 02:20:58.904839 13437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:20:58.905786 13437 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 02:20:58.908819 13437 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 02:20:58.914492 13437 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:20:58.915545 13437 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b: Loaded TSK: 0
I20260808 02:20:58.916813 13437 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 02:20:58.990485 13422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:20:58.990805 13422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:20:58.990875 13422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:20:58.995594 13422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:20:58.995719 13422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:20:58.995808 13422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:20:59.002501 13422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:46371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:34449
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=34277
--tserver_master_addrs=127.24.79.254:43955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:20:59.003875 13422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:20:59.005079 13422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:20:59.018157 13454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:59.018558 13422 server_base.cc:1034] running on GCE node
W20260808 02:20:59.018712 13455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:59.018154 13461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:59.019418 13422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:20:59.020630 13422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:20:59.021893 13422 hybrid_clock.cc:648] HybridClock initialized: now 1786155659021842 us; error 64 us; skew 500 ppm
Aug 08 02:20:59 dist-test-slave-1vmg krb5kdc[12549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155659, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:20:59.026552 13422 init.cc:377] Logged in from keytab as kudu/127.24.79.193@KRBTEST.COM (short username kudu)
I20260808 02:20:59.029026 13422 webserver.cc:460] Webserver started at http://127.24.79.193:34277/ using document root <none> and password file <none>
I20260808 02:20:59.029992 13422 fs_manager.cc:359] Metadata directory not provided
I20260808 02:20:59.030149 13422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:20:59.036746 13422 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260808 02:20:59.040186 13471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:59.041961 13422 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:20:59.042127 13422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "1b385bef71934d1e9c4e8db6120530fb"
format_stamp: "Formatted at 2026-08-08 02:20:57 on dist-test-slave-1vmg"
server_key: "0878b4f99b348c58f2ffffe5cd416dfd"
server_key_iv: "80d778bc23ee007807ea0b2a390cdc63"
server_key_version: "encryptionkey@0"
I20260808 02:20:59.042959 13422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:20:59.065411 13422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:20:59.070266 13422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:20:59.070539 13422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:20:59.071431 13422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:20:59.072765 13486 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 02:20:59.078099 13422 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 02:20:59.078225 13422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260808 02:20:59.078321 13422 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 02:20:59.080772 13422 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 02:20:59.080854 13422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.000s
I20260808 02:20:59.081241 13486 tablet_bootstrap.cc:492] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb: Bootstrap starting.
I20260808 02:20:59.092337 13422 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:34449
I20260808 02:20:59.092417 13634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:34449 every 8 connection(s)
I20260808 02:20:59.093555 13422 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:20:59.100554 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 13422
I20260808 02:20:59.101065 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:34687
--local_ip_for_outbound_sockets=127.24.79.194
--tserver_master_addrs=127.24.79.254:43955
--webserver_port=35199
--webserver_interface=127.24.79.194
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:46371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:20:59.102699 13486 log.cc:824] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:20:59 dist-test-slave-1vmg krb5kdc[12549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155659, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:20:59.113277 13486 tablet_bootstrap.cc:492] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:20:59.113940 13486 tablet_bootstrap.cc:492] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb: Bootstrap complete.
I20260808 02:20:59.114517 13639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:59.096168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:40143 (local address 127.24.79.254:43955)
0808 02:20:59.096776 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:59.096804 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:59.096990 (+ 186us) server_negotiation.cc:407] Connection header received
0808 02:20:59.098632 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:59.098650 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:59.099246 (+ 596us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:59.099792 (+ 546us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:59.101253 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.102729 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.104658 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.105465 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.105697 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:59.108605 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:59.108611 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:59.108627 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:59.108665 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:59.110658 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:59.111393 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:59.111399 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:59.111404 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:59.111560 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:59.111964 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:59.111967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:59.111968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:59.112410 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:59.112557 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:59.113234 (+ 677us) server_negotiation.cc:299] Negotiation successful
0808 02:20:59.113558 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":169,"threads_started":1}
I20260808 02:20:59.114518 13640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:59.096129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.193:40143)
0808 02:20:59.096779 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:59.096843 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:59.098020 (+ 1177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:59.099865 (+ 1845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:59.099874 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:59.100382 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:20:59.101007 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:59.101028 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.102999 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.103004 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:59.104448 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:59.104466 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.105686 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.105690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:59.105869 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:59.106576 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:20:59.106601 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:20:59.108321 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:20:59.110839 (+ 2518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:59.110847 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:59.110865 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:59.111245 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:59.111680 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:59.111683 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:59.111686 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:59.111845 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:59.112543 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:20:59.112550 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:20:59.112839 (+ 289us) client_negotiation.cc:769] Sending connection context
0808 02:20:59.113031 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 02:20:59.113459 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":218,"threads_started":1}
I20260808 02:20:59.115787 13486 ts_tablet_manager.cc:1397] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb: Time spent bootstrapping tablet: real 0.035s user 0.026s sys 0.008s
I20260808 02:20:59.117710 13635 heartbeater.cc:347] Connected to a master server at 127.24.79.254:43955
I20260808 02:20:59.117981 13635 heartbeater.cc:464] Registering TS with master...
I20260808 02:20:59.118549 13635 heartbeater.cc:511] Master 127.24.79.254:43955 requested a full tablet report, sending...
I20260808 02:20:59.119289 13486 raft_consensus.cc:348] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:20:59.119925 13486 raft_consensus.cc:729] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b385bef71934d1e9c4e8db6120530fb, State: Initialized, Role: FOLLOWER
I20260808 02:20:59.120518 13486 consensus_queue.cc:260] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:20:59.121670 13486 ts_tablet_manager.cc:1428] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 02:20:59.121691 13368 ts_manager.cc:194] Registered new tserver with Master: 1b385bef71934d1e9c4e8db6120530fb (127.24.79.193:34449)
I20260808 02:20:59.125964 13368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.193@KRBTEST.COM'} at 127.24.79.193:40143
I20260808 02:20:59.129385 13635 heartbeater.cc:503] Master 127.24.79.254:43955 was elected leader, sending a full tablet report...
Aug 08 02:20:59 dist-test-slave-1vmg krb5kdc[12549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155658, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for kudu/127.24.79.193@KRBTEST.COM
I20260808 02:20:59.139070 13645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:59.125420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:34449 (local address 127.0.0.1:53794)
0808 02:20:59.125971 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:59.126038 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:59.126200 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:59.127684 (+ 1484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:59.127692 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:59.128191 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:20:59.128359 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:59.128368 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.129809 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.129813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:59.130702 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:59.130717 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.131622 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.131627 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:59.131741 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:59.132360 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:20:59.132374 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:20:59.134137 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:20:59.136698 (+ 2561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:59.136714 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:59.136720 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:59.137069 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:59.137506 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:59.137509 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:59.137510 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:59.137626 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:59.138314 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:20:59.138320 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:20:59.138465 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 02:20:59.138605 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 02:20:59.138838 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":148,"threads_started":1}
I20260808 02:20:59.139191 13646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:59.125517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53794 (local address 127.24.79.193:34449)
0808 02:20:59.127142 (+ 1625us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:59.127153 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:59.127178 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:20:59.127249 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:59.127254 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:59.127489 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:59.127645 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:59.128521 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.129663 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.130861 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.131454 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.131570 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:59.134338 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:59.134355 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:59.134370 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:59.134406 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:59.136498 (+ 2092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:59.137219 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:59.137227 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:59.137229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:59.137349 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:59.137735 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:59.137738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:59.137739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:59.138114 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:59.138269 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:59.138734 (+ 465us) server_negotiation.cc:299] Negotiation successful
0808 02:20:59.139003 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1503,"spinlock_wait_cycles":1408,"thread_start_us":127,"threads_started":1}
I20260808 02:20:59.140440 13537 tablet_service.cc:1514] Processing DeleteTablet for tablet 55a2be20a9e34dd4a45e2e974b4f9bd1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:20:58 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:53794
I20260808 02:20:59.141417 13648 tablet_replica.cc:330] stopping tablet replica
I20260808 02:20:59.141770 13648 raft_consensus.cc:2229] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:20:59.142037 13648 raft_consensus.cc:2258] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:20:59.143416 13648 ts_tablet_manager.cc:1905] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:20:59.148077 13648 ts_tablet_manager.cc:1918] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:20:59.148249 13648 log.cc:1196] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/wal/wals/55a2be20a9e34dd4a45e2e974b4f9bd1
I20260808 02:20:59.148630 13648 ts_tablet_manager.cc:1939] T 55a2be20a9e34dd4a45e2e974b4f9bd1 P 1b385bef71934d1e9c4e8db6120530fb: Deleting consensus metadata
I20260808 02:20:59.150808 13350 catalog_manager.cc:4841] TS 1b385bef71934d1e9c4e8db6120530fb (127.24.79.193:34449): tablet 55a2be20a9e34dd4a45e2e974b4f9bd1 (table test-table [id=de3b5c3d31c84ca1b13b5d0d67199e8b]) successfully deleted
W20260808 02:20:59.240658 13641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:20:59.241040 13641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:20:59.241138 13641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:20:59.245471 13641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:20:59.245568 13641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:20:59.245632 13641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:20:59.251694 13641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:46371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:34687
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=35199
--tserver_master_addrs=127.24.79.254:43955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:20:59.253005 13641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:20:59.254124 13641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:20:59.265509 13655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:59.265755 13641 server_base.cc:1034] running on GCE node
W20260808 02:20:59.265592 13656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:59.265532 13658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:59.266350 13641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:20:59.267159 13641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:20:59.268365 13641 hybrid_clock.cc:648] HybridClock initialized: now 1786155659268345 us; error 38 us; skew 500 ppm
Aug 08 02:20:59 dist-test-slave-1vmg krb5kdc[12549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155659, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:20:59.271797 13641 init.cc:377] Logged in from keytab as kudu/127.24.79.194@KRBTEST.COM (short username kudu)
I20260808 02:20:59.272971 13641 webserver.cc:460] Webserver started at http://127.24.79.194:35199/ using document root <none> and password file <none>
I20260808 02:20:59.273687 13641 fs_manager.cc:359] Metadata directory not provided
I20260808 02:20:59.273793 13641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:20:59.279075 13641 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:20:59.281738 13667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:59.283638 13641 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:20:59.283823 13641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "195a12ab501c46058d4d4733c3cf3835"
format_stamp: "Formatted at 2026-08-08 02:20:58 on dist-test-slave-1vmg"
server_key: "8235a3b1041433774ce3788405e123b0"
server_key_iv: "705a8737909dfee1ab0f9ef5cf8ad11d"
server_key_version: "encryptionkey@0"
I20260808 02:20:59.284502 13641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:20:59.329360 13641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:20:59.334713 13641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:20:59.334956 13641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:20:59.335690 13641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:20:59.336704 13641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:20:59.336763 13641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:59.336810 13641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:20:59.336824 13641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:59.347541 13641 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:34687
I20260808 02:20:59.347755 13819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:34687 every 8 connection(s)
I20260808 02:20:59.348809 13641 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:20:59.353272 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 13641
I20260808 02:20:59.353960 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:41277
--local_ip_for_outbound_sockets=127.24.79.195
--tserver_master_addrs=127.24.79.254:43955
--webserver_port=38049
--webserver_interface=127.24.79.195
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:46371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:20:59 dist-test-slave-1vmg krb5kdc[12549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155659, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:20:59.371567 13639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:59.351373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:34909 (local address 127.24.79.254:43955)
0808 02:20:59.351547 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:59.351553 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:59.352890 (+ 1337us) server_negotiation.cc:407] Connection header received
0808 02:20:59.356609 (+ 3719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:59.356614 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:59.356695 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:59.356810 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:59.358892 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.359817 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.361853 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.362260 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.362365 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:59.366394 (+ 4029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:59.366399 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:59.366403 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:59.366444 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:59.368357 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:59.369272 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:59.369277 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:59.369279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:59.369412 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:59.369929 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:59.369932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:59.369933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:59.370186 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:59.370288 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:59.371182 (+ 894us) server_negotiation.cc:299] Negotiation successful
0808 02:20:59.371375 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:20:59.372685 13823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:59.351894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.194:34909)
0808 02:20:59.352663 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:59.352729 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:59.356149 (+ 3420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:59.357105 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:59.357119 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:59.357789 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:20:59.358603 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:59.358628 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.360027 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.360031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:59.361613 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:59.361633 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.362661 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.362666 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:59.363009 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:59.363852 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:20:59.363880 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:20:59.366075 (+ 2195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:20:59.368541 (+ 2466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:59.368552 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:59.368580 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:59.369110 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:59.369544 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:59.369548 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:59.369550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:59.369823 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:59.370356 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:20:59.370365 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:20:59.370769 (+ 404us) client_negotiation.cc:769] Sending connection context
0808 02:20:59.371074 (+ 305us) client_negotiation.cc:240] Negotiation successful
0808 02:20:59.371498 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":298,"threads_started":1}
I20260808 02:20:59.375905 13820 heartbeater.cc:347] Connected to a master server at 127.24.79.254:43955
I20260808 02:20:59.376366 13820 heartbeater.cc:464] Registering TS with master...
I20260808 02:20:59.377336 13820 heartbeater.cc:511] Master 127.24.79.254:43955 requested a full tablet report, sending...
I20260808 02:20:59.378906 13368 ts_manager.cc:194] Registered new tserver with Master: 195a12ab501c46058d4d4733c3cf3835 (127.24.79.194:34687)
I20260808 02:20:59.379873 13368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.194@KRBTEST.COM'} at 127.24.79.194:34909
W20260808 02:20:59.500770 13824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:20:59.501185 13824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:20:59.501276 13824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:20:59.505966 13824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:20:59.506093 13824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:20:59.506187 13824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:20:59.512490 13824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:46371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:41277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=38049
--tserver_master_addrs=127.24.79.254:43955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:20:59.513790 13824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:20:59.515121 13824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:20:59.527153 13835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:20:59.527158 13830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:59.527354 13824 server_base.cc:1034] running on GCE node
W20260808 02:20:59.527153 13833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:20:59.528000 13824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:20:59.528801 13824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:20:59.530046 13824 hybrid_clock.cc:648] HybridClock initialized: now 1786155659530020 us; error 43 us; skew 500 ppm
Aug 08 02:20:59 dist-test-slave-1vmg krb5kdc[12549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155659, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:20:59.533502 13824 init.cc:377] Logged in from keytab as kudu/127.24.79.195@KRBTEST.COM (short username kudu)
I20260808 02:20:59.534687 13824 webserver.cc:460] Webserver started at http://127.24.79.195:38049/ using document root <none> and password file <none>
I20260808 02:20:59.535405 13824 fs_manager.cc:359] Metadata directory not provided
I20260808 02:20:59.535509 13824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:20:59.540371 13824 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:20:59.543253 13846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:59.544873 13824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:20:59.545024 13824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "204278d3b4cb446c932dba11b4dff15e"
format_stamp: "Formatted at 2026-08-08 02:20:58 on dist-test-slave-1vmg"
server_key: "176f6b683a4898042d84e3d4e91a9991"
server_key_iv: "6b63bf09a10b11ff9ceb5497ef5f5809"
server_key_version: "encryptionkey@0"
I20260808 02:20:59.545635 13824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:20:59.564734 13824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:20:59.568516 13824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:20:59.568742 13824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:20:59.569437 13824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:20:59.570432 13824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:20:59.570488 13824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:59.570550 13824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:20:59.570580 13824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:20:59.581084 13824 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:41277
I20260808 02:20:59.581112 14019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:41277 every 8 connection(s)
I20260808 02:20:59.582276 13824 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:20:59.585856 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 13824
Aug 08 02:20:59 dist-test-slave-1vmg krb5kdc[12549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155659, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:20:59.596056 14025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:59.586666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45450 (local address 127.24.79.254:43955)
0808 02:20:59.586909 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:59.586939 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:59.587165 (+ 226us) server_negotiation.cc:407] Connection header received
0808 02:20:59.587327 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:59.587330 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:59.587380 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:59.587464 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:59.588278 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.588835 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.589852 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.590088 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.590181 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:59.591799 (+ 1618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:59.591802 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:59.591804 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:59.591832 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:59.593615 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:59.594173 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:59.594178 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:59.594180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:59.594305 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:59.594676 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:59.594681 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:59.594683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:59.595073 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:59.595369 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:59.595661 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 02:20:59.595829 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"spinlock_wait_cycles":35840,"thread_start_us":72,"threads_started":1}
I20260808 02:20:59.599247 13639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:59.584541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:58857 (local address 127.24.79.254:43955)
0808 02:20:59.584705 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:59.584710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:59.585758 (+ 1048us) server_negotiation.cc:407] Connection header received
0808 02:20:59.587615 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:59.587618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:59.587683 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:59.587758 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:20:59.589329 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.589873 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.591384 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.591622 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.591725 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:59.594787 (+ 3062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:20:59.594790 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:20:59.594792 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:20:59.594849 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:20:59.596613 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:59.597338 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:59.597341 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:59.597343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:59.597444 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:20:59.597892 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:20:59.597896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:20:59.597898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:20:59.598147 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:20:59.598248 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:59.598899 (+ 651us) server_negotiation.cc:299] Negotiation successful
0808 02:20:59.599079 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59,"spinlock_wait_cycles":4992}
I20260808 02:20:59.600229 14023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:59.584949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.195:58857)
0808 02:20:59.585536 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:59.585594 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:59.587349 (+ 1755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:59.587984 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:59.587994 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:59.588463 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:20:59.589127 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:59.589151 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.590030 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.590032 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:59.591244 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:59.591256 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.591811 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.591815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:59.591992 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:59.592577 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:20:59.592596 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:20:59.594571 (+ 1975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:20:59.596777 (+ 2206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:59.596786 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:59.596809 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:59.597216 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:59.597596 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:20:59.597599 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:20:59.597601 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:20:59.597788 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:20:59.598322 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:20:59.598329 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:20:59.598626 (+ 297us) client_negotiation.cc:769] Sending connection context
0808 02:20:59.598831 (+ 205us) client_negotiation.cc:240] Negotiation successful
0808 02:20:59.599268 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":171,"threads_started":1}
I20260808 02:20:59.601706 14020 heartbeater.cc:347] Connected to a master server at 127.24.79.254:43955
I20260808 02:20:59.601979 14020 heartbeater.cc:464] Registering TS with master...
I20260808 02:20:59.602566 14020 heartbeater.cc:511] Master 127.24.79.254:43955 requested a full tablet report, sending...
I20260808 02:20:59.603847 13368 ts_manager.cc:194] Registered new tserver with Master: 204278d3b4cb446c932dba11b4dff15e (127.24.79.195:41277)
I20260808 02:20:59.604480 13368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.195@KRBTEST.COM'} at 127.24.79.195:58857
I20260808 02:20:59.610641 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:20:59.616164 13639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:59.611420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45452 (local address 127.24.79.254:43955)
0808 02:20:59.611571 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:59.611576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:59.611767 (+ 191us) server_negotiation.cc:407] Connection header received
0808 02:20:59.611959 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:59.611962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:59.612080 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:59.612211 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:20:59.612477 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.613011 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.614074 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.614335 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.614435 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:59.614965 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:20:59.615679 (+ 714us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:20:59.615837 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:59.615921 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 02:20:59.615994 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 02:20:59.617128 13368 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:45452:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:20:59.620939 13368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:20:59.631089 13537 tablet_service.cc:1467] Processing CreateTablet for tablet e7fc511944f04f748270034c95b9ae45 (DEFAULT_TABLE table=test-table [id=8a4eea6e5e954ba3aba11c9d80cd9fe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:20:59.631877 13537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7fc511944f04f748270034c95b9ae45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:20:59.635421 13645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:59.629602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:41277 (local address 127.0.0.1:37790)
0808 02:20:59.629773 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:59.629792 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:59.629899 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:59.631101 (+ 1202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:59.631104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:59.631126 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:20:59.631264 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:59.631272 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.632828 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.632832 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:59.633909 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:59.633922 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.634879 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.634883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:59.635021 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:59.635086 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 02:20:59.635149 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 02:20:59.635274 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 02:20:59.635504 14032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:59.629926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37790 (local address 127.24.79.195:41277)
0808 02:20:59.630454 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:59.630465 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:59.630490 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:20:59.630584 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:59.630590 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:59.630862 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:59.631117 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:20:59.631432 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.632654 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.634078 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.634730 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.634877 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:59.635116 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:59.635213 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 02:20:59.635333 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":155,"threads_started":1}
I20260808 02:20:59.635921 14036 tablet_bootstrap.cc:492] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb: Bootstrap starting.
I20260808 02:20:59.637069 14036 tablet_bootstrap.cc:654] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb: Neither blocks nor log segments found. Creating new log.
I20260808 02:20:59.638427 14036 tablet_bootstrap.cc:492] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb: No bootstrap required, opened a new log
I20260808 02:20:59.638547 14036 ts_tablet_manager.cc:1397] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:20:59.639060 14036 raft_consensus.cc:348] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:20:59.639187 14036 raft_consensus.cc:374] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:20:59.639235 14036 raft_consensus.cc:729] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b385bef71934d1e9c4e8db6120530fb, State: Initialized, Role: FOLLOWER
I20260808 02:20:59.639436 14036 consensus_queue.cc:260] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:20:59.639887 14034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:59.630338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58806 (local address 127.24.79.194:34687)
0808 02:20:59.630845 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:59.630867 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:59.630892 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:20:59.631060 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:59.631067 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:59.631266 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:59.631427 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:20:59.631806 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.633311 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.634771 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.635731 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.636061 (+ 330us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:59.636235 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:59.639520 (+ 3285us) server_negotiation.cc:299] Negotiation successful
0808 02:20:59.639684 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":167,"threads_started":1}
I20260808 02:20:59.639895 14036 ts_tablet_manager.cc:1428] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 02:20:59.639904 14033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:59.630059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:34687 (local address 127.0.0.1:58806)
0808 02:20:59.630768 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:59.630785 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:59.630892 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:59.631525 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:59.631528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:59.631549 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:20:59.631663 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:59.631670 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.633465 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.633469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:59.634589 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:59.634601 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.639058 (+ 4457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.639063 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:59.639202 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:59.639238 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 02:20:59.639409 (+ 171us) client_negotiation.cc:240] Negotiation successful
0808 02:20:59.639731 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":100,"threads_started":1}
I20260808 02:20:59.639467 13915 tablet_service.cc:1467] Processing CreateTablet for tablet e7fc511944f04f748270034c95b9ae45 (DEFAULT_TABLE table=test-table [id=8a4eea6e5e954ba3aba11c9d80cd9fe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:20:59.640895 13915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7fc511944f04f748270034c95b9ae45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:20:59.644119 13731 tablet_service.cc:1467] Processing CreateTablet for tablet e7fc511944f04f748270034c95b9ae45 (DEFAULT_TABLE table=test-table [id=8a4eea6e5e954ba3aba11c9d80cd9fe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:20:59.645444 13731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7fc511944f04f748270034c95b9ae45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:20:59.650381 14039 tablet_bootstrap.cc:492] T e7fc511944f04f748270034c95b9ae45 P 204278d3b4cb446c932dba11b4dff15e: Bootstrap starting.
I20260808 02:20:59.653510 14039 tablet_bootstrap.cc:654] T e7fc511944f04f748270034c95b9ae45 P 204278d3b4cb446c932dba11b4dff15e: Neither blocks nor log segments found. Creating new log.
I20260808 02:20:59.654263 14041 tablet_bootstrap.cc:492] T e7fc511944f04f748270034c95b9ae45 P 195a12ab501c46058d4d4733c3cf3835: Bootstrap starting.
I20260808 02:20:59.654644 14039 log.cc:824] T e7fc511944f04f748270034c95b9ae45 P 204278d3b4cb446c932dba11b4dff15e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:20:59.657212 14039 tablet_bootstrap.cc:492] T e7fc511944f04f748270034c95b9ae45 P 204278d3b4cb446c932dba11b4dff15e: No bootstrap required, opened a new log
I20260808 02:20:59.657437 14041 tablet_bootstrap.cc:654] T e7fc511944f04f748270034c95b9ae45 P 195a12ab501c46058d4d4733c3cf3835: Neither blocks nor log segments found. Creating new log.
I20260808 02:20:59.657488 14039 ts_tablet_manager.cc:1397] T e7fc511944f04f748270034c95b9ae45 P 204278d3b4cb446c932dba11b4dff15e: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 02:20:59.658433 14041 log.cc:824] T e7fc511944f04f748270034c95b9ae45 P 195a12ab501c46058d4d4733c3cf3835: Log is configured to *not* fsync() on all Append() calls
I20260808 02:20:59.661194 14041 tablet_bootstrap.cc:492] T e7fc511944f04f748270034c95b9ae45 P 195a12ab501c46058d4d4733c3cf3835: No bootstrap required, opened a new log
I20260808 02:20:59.661480 14041 ts_tablet_manager.cc:1397] T e7fc511944f04f748270034c95b9ae45 P 195a12ab501c46058d4d4733c3cf3835: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 02:20:59.661741 14039 raft_consensus.cc:348] T e7fc511944f04f748270034c95b9ae45 P 204278d3b4cb446c932dba11b4dff15e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:20:59.662079 14039 raft_consensus.cc:374] T e7fc511944f04f748270034c95b9ae45 P 204278d3b4cb446c932dba11b4dff15e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:20:59.662144 14039 raft_consensus.cc:729] T e7fc511944f04f748270034c95b9ae45 P 204278d3b4cb446c932dba11b4dff15e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 204278d3b4cb446c932dba11b4dff15e, State: Initialized, Role: FOLLOWER
I20260808 02:20:59.662691 14039 consensus_queue.cc:260] T e7fc511944f04f748270034c95b9ae45 P 204278d3b4cb446c932dba11b4dff15e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:20:59.663585 14020 heartbeater.cc:503] Master 127.24.79.254:43955 was elected leader, sending a full tablet report...
I20260808 02:20:59.663918 14039 ts_tablet_manager.cc:1428] T e7fc511944f04f748270034c95b9ae45 P 204278d3b4cb446c932dba11b4dff15e: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260808 02:20:59.665501 14041 raft_consensus.cc:348] T e7fc511944f04f748270034c95b9ae45 P 195a12ab501c46058d4d4733c3cf3835 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:20:59.665787 14041 raft_consensus.cc:374] T e7fc511944f04f748270034c95b9ae45 P 195a12ab501c46058d4d4733c3cf3835 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:20:59.665855 14041 raft_consensus.cc:729] T e7fc511944f04f748270034c95b9ae45 P 195a12ab501c46058d4d4733c3cf3835 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 195a12ab501c46058d4d4733c3cf3835, State: Initialized, Role: FOLLOWER
I20260808 02:20:59.666324 14041 consensus_queue.cc:260] T e7fc511944f04f748270034c95b9ae45 P 195a12ab501c46058d4d4733c3cf3835 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:20:59.667140 13820 heartbeater.cc:503] Master 127.24.79.254:43955 was elected leader, sending a full tablet report...
I20260808 02:20:59.667515 14041 ts_tablet_manager.cc:1428] T e7fc511944f04f748270034c95b9ae45 P 195a12ab501c46058d4d4733c3cf3835: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260808 02:20:59.744284 14038 raft_consensus.cc:482] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:20:59.744521 14038 raft_consensus.cc:504] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:20:59.745750 14038 leader_election.cc:288] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 204278d3b4cb446c932dba11b4dff15e (127.24.79.195:41277), 195a12ab501c46058d4d4733c3cf3835 (127.24.79.194:34687)
I20260808 02:20:59.750309 14034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:59.746195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:60143 (local address 127.24.79.194:34687)
0808 02:20:59.746383 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:59.746388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:59.746601 (+ 213us) server_negotiation.cc:407] Connection header received
0808 02:20:59.746780 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:59.746784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:59.746843 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:59.746932 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:20:59.747310 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.747841 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.749183 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.749665 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.749782 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:59.749868 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:59.750092 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 02:20:59.750179 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260808 02:20:59.750326 14047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:59.746078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:34687 (local address 127.24.79.193:60143)
0808 02:20:59.746500 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:59.746512 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:59.746640 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:59.746965 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:59.746968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:59.746985 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:20:59.747196 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:59.747205 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.747944 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.747948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:59.749025 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:59.749036 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.749795 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.749798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:59.749899 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:59.749975 (+ 76us) client_negotiation.cc:769] Sending connection context
0808 02:20:59.750047 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 02:20:59.750157 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":1,"thread_start_us":134,"threads_started":1}
I20260808 02:20:59.750309 14032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:59.746247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:33235 (local address 127.24.79.195:41277)
0808 02:20:59.746382 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:59.746387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:59.746719 (+ 332us) server_negotiation.cc:407] Connection header received
0808 02:20:59.746903 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:59.746907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:59.746994 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:59.747081 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:20:59.747367 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.747936 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.749178 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.749698 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.749835 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:59.749891 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:59.750085 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 02:20:59.750164 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 02:20:59.750329 14046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:59.746081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:41277 (local address 127.24.79.193:33235)
0808 02:20:59.746604 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:59.746619 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:20:59.746747 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:20:59.747097 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:20:59.747100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:20:59.747115 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:20:59.747254 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:59.747260 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.748126 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.748129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:59.749013 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:20:59.749025 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.749823 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.749827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:20:59.749934 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:59.749982 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 02:20:59.750062 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 02:20:59.750152 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"mutex_wait_us":64,"thread_start_us":109,"threads_started":1}
I20260808 02:20:59.751138 13951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7fc511944f04f748270034c95b9ae45" candidate_uuid: "1b385bef71934d1e9c4e8db6120530fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204278d3b4cb446c932dba11b4dff15e" is_pre_election: true
I20260808 02:20:59.751140 13757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7fc511944f04f748270034c95b9ae45" candidate_uuid: "1b385bef71934d1e9c4e8db6120530fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "195a12ab501c46058d4d4733c3cf3835" is_pre_election: true
I20260808 02:20:59.751477 13757 raft_consensus.cc:2454] T e7fc511944f04f748270034c95b9ae45 P 195a12ab501c46058d4d4733c3cf3835 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b385bef71934d1e9c4e8db6120530fb in term 0.
I20260808 02:20:59.751477 13951 raft_consensus.cc:2454] T e7fc511944f04f748270034c95b9ae45 P 204278d3b4cb446c932dba11b4dff15e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b385bef71934d1e9c4e8db6120530fb in term 0.
I20260808 02:20:59.751976 13477 leader_election.cc:302] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b385bef71934d1e9c4e8db6120530fb, 204278d3b4cb446c932dba11b4dff15e; no voters:
I20260808 02:20:59.752310 14038 raft_consensus.cc:2781] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:20:59.752399 14038 raft_consensus.cc:482] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:20:59.752431 14038 raft_consensus.cc:3037] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:20:59.753587 14038 raft_consensus.cc:504] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:20:59.754027 14038 leader_election.cc:288] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [CANDIDATE]: Term 1 election: Requested vote from peers 204278d3b4cb446c932dba11b4dff15e (127.24.79.195:41277), 195a12ab501c46058d4d4733c3cf3835 (127.24.79.194:34687)
I20260808 02:20:59.754563 13757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7fc511944f04f748270034c95b9ae45" candidate_uuid: "1b385bef71934d1e9c4e8db6120530fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "195a12ab501c46058d4d4733c3cf3835"
I20260808 02:20:59.754606 13951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7fc511944f04f748270034c95b9ae45" candidate_uuid: "1b385bef71934d1e9c4e8db6120530fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204278d3b4cb446c932dba11b4dff15e"
I20260808 02:20:59.754704 13757 raft_consensus.cc:3037] T e7fc511944f04f748270034c95b9ae45 P 195a12ab501c46058d4d4733c3cf3835 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:20:59.754760 13951 raft_consensus.cc:3037] T e7fc511944f04f748270034c95b9ae45 P 204278d3b4cb446c932dba11b4dff15e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:20:59.755968 13757 raft_consensus.cc:2454] T e7fc511944f04f748270034c95b9ae45 P 195a12ab501c46058d4d4733c3cf3835 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b385bef71934d1e9c4e8db6120530fb in term 1.
I20260808 02:20:59.755975 13951 raft_consensus.cc:2454] T e7fc511944f04f748270034c95b9ae45 P 204278d3b4cb446c932dba11b4dff15e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b385bef71934d1e9c4e8db6120530fb in term 1.
I20260808 02:20:59.756388 13482 leader_election.cc:302] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 195a12ab501c46058d4d4733c3cf3835, 1b385bef71934d1e9c4e8db6120530fb, 204278d3b4cb446c932dba11b4dff15e; no voters:
I20260808 02:20:59.756604 14038 raft_consensus.cc:2781] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:20:59.756827 14038 raft_consensus.cc:686] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 1 LEADER]: Becoming Leader. State: Replica: 1b385bef71934d1e9c4e8db6120530fb, State: Running, Role: LEADER
I20260808 02:20:59.757138 14038 consensus_queue.cc:237] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:20:59.761086 13368 catalog_manager.cc:5495] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b385bef71934d1e9c4e8db6120530fb (127.24.79.193). New cstate: current_term: 1 leader_uuid: "1b385bef71934d1e9c4e8db6120530fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } health_report { overall_health: HEALTHY } } }
I20260808 02:20:59.798177 14053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:59.793488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53808 (local address 127.24.79.193:34449)
0808 02:20:59.793883 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0808 02:20:59.793890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:20:59.793907 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:20:59.794001 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:20:59.794004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:20:59.794092 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:20:59.794230 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:20:59.794574 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.795235 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.796197 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:20:59.796486 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:20:59.796581 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:20:59.796932 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:20:59.797502 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:20:59.797605 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 02:20:59.797743 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 02:20:59.797812 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":139,"threads_started":1}
I20260808 02:20:59.803223 13757 raft_consensus.cc:1261] T e7fc511944f04f748270034c95b9ae45 P 195a12ab501c46058d4d4733c3cf3835 [term 1 FOLLOWER]: Refusing update from remote peer 1b385bef71934d1e9c4e8db6120530fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:20:59.803223 13951 raft_consensus.cc:1261] T e7fc511944f04f748270034c95b9ae45 P 204278d3b4cb446c932dba11b4dff15e [term 1 FOLLOWER]: Refusing update from remote peer 1b385bef71934d1e9c4e8db6120530fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:20:59.804306 14049 consensus_queue.cc:1035] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:20:59.804574 14038 consensus_queue.cc:1035] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:20:59.816143 14057 mvcc.cc:205] Tried to move back new op lower bound from 7316093582545342464 to 7316093582368739328. Current Snapshot: MvccSnapshot[applied={T|T < 7316093582545342464}]
I20260808 02:20:59.824918 13366 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:45452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:20:59.825155 13366 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:45452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:20:59.828454 13366 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b: Sending DeleteTablet for 3 replicas of tablet e7fc511944f04f748270034c95b9ae45
I20260808 02:20:59.829577 13537 tablet_service.cc:1514] Processing DeleteTablet for tablet e7fc511944f04f748270034c95b9ae45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:20:59 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:53794
I20260808 02:20:59.829525 13731 tablet_service.cc:1514] Processing DeleteTablet for tablet e7fc511944f04f748270034c95b9ae45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:20:59 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:58806
I20260808 02:20:59.829667 13915 tablet_service.cc:1514] Processing DeleteTablet for tablet e7fc511944f04f748270034c95b9ae45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:20:59 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:37790
I20260808 02:20:59.830060 14067 tablet_replica.cc:330] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb: stopping tablet replica
I20260808 02:20:59.830209 14067 raft_consensus.cc:2229] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:20:59.830542 14068 tablet_replica.cc:330] T e7fc511944f04f748270034c95b9ae45 P 204278d3b4cb446c932dba11b4dff15e: stopping tablet replica
I20260808 02:20:59.830575 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 13422
I20260808 02:20:59.830631 14067 raft_consensus.cc:2258] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:20:59.845007 14071 tablet_replica.cc:330] T e7fc511944f04f748270034c95b9ae45 P 195a12ab501c46058d4d4733c3cf3835: stopping tablet replica
I20260808 02:20:59.845860 14071 raft_consensus.cc:2229] T e7fc511944f04f748270034c95b9ae45 P 195a12ab501c46058d4d4733c3cf3835 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:20:59.846454 14071 raft_consensus.cc:2258] T e7fc511944f04f748270034c95b9ae45 P 195a12ab501c46058d4d4733c3cf3835 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:20:59.848453 14071 ts_tablet_manager.cc:1905] T e7fc511944f04f748270034c95b9ae45 P 195a12ab501c46058d4d4733c3cf3835: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:20:59.852160 14068 raft_consensus.cc:2229] T e7fc511944f04f748270034c95b9ae45 P 204278d3b4cb446c932dba11b4dff15e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:20:59.852640 14071 ts_tablet_manager.cc:1918] T e7fc511944f04f748270034c95b9ae45 P 195a12ab501c46058d4d4733c3cf3835: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:20:59.852854 14071 log.cc:1196] T e7fc511944f04f748270034c95b9ae45 P 195a12ab501c46058d4d4733c3cf3835: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/e7fc511944f04f748270034c95b9ae45
I20260808 02:20:59.852967 14068 raft_consensus.cc:2258] T e7fc511944f04f748270034c95b9ae45 P 204278d3b4cb446c932dba11b4dff15e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:20:59.853236 14071 ts_tablet_manager.cc:1939] T e7fc511944f04f748270034c95b9ae45 P 195a12ab501c46058d4d4733c3cf3835: Deleting consensus metadata
I20260808 02:20:59.855137 14068 ts_tablet_manager.cc:1905] T e7fc511944f04f748270034c95b9ae45 P 204278d3b4cb446c932dba11b4dff15e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 02:20:59.857823 13350 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.79.193:34449 (error 108)
I20260808 02:20:59.858359 14068 ts_tablet_manager.cc:1918] T e7fc511944f04f748270034c95b9ae45 P 204278d3b4cb446c932dba11b4dff15e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:20:59.858503 14068 log.cc:1196] T e7fc511944f04f748270034c95b9ae45 P 204278d3b4cb446c932dba11b4dff15e: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/e7fc511944f04f748270034c95b9ae45
I20260808 02:20:59.858850 14068 ts_tablet_manager.cc:1939] T e7fc511944f04f748270034c95b9ae45 P 204278d3b4cb446c932dba11b4dff15e: Deleting consensus metadata
I20260808 02:20:59.859259 14033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:59.858871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:34449 (local address 127.0.0.1:53816)
0808 02:20:59.859099 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:59.859164 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.193:34449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
W20260808 02:20:59.859549 13350 catalog_manager.cc:4584] TS 1b385bef71934d1e9c4e8db6120530fb (127.24.79.193:34449): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e7fc511944f04f748270034c95b9ae45: Network error: Client connection negotiation failed: client connection to 127.24.79.193:34449: connect: Connection refused (error 111)
I20260808 02:20:59.860128 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 13641
I20260808 02:20:59.867744 13349 catalog_manager.cc:4841] TS 204278d3b4cb446c932dba11b4dff15e (127.24.79.195:41277): tablet e7fc511944f04f748270034c95b9ae45 (table test-table [id=8a4eea6e5e954ba3aba11c9d80cd9fe9]) successfully deleted
I20260808 02:20:59.874534 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 13824
I20260808 02:20:59.877771 14033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:20:59.874891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:34687 (local address 127.0.0.1:58816)
0808 02:20:59.877574 (+ 2683us) negotiation.cc:107] Waiting for socket to connect
0808 02:20:59.877663 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.194:34687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2583}
I20260808 02:20:59.886543 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 13321
I20260808 02:20:59.902987 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:43955
--webserver_interface=127.24.79.254
--webserver_port=42591
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:46371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:43955
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:21:00.035604 14073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:21:00.035951 14073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:21:00.035995 14073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:21:00.040411 14073 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:21:00.040518 14073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:21:00.040540 14073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:21:00.040556 14073 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:21:00.040571 14073 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:21:00.047523 14073 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:46371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:43955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:43955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=42591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:21:00.048900 14073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:21:00.050161 14073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:21:00.061117 14073 server_base.cc:1034] running on GCE node
W20260808 02:21:00.061046 14079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:21:00.061044 14080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:21:00.061046 14082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:21:00.062183 14073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:21:00.063621 14073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:21:00.064877 14073 hybrid_clock.cc:648] HybridClock initialized: now 1786155660064844 us; error 55 us; skew 500 ppm
Aug 08 02:21:00 dist-test-slave-1vmg krb5kdc[12549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155660, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:21:00.068645 14073 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:21:00.069869 14073 webserver.cc:460] Webserver started at http://127.24.79.254:42591/ using document root <none> and password file <none>
I20260808 02:21:00.070564 14073 fs_manager.cc:359] Metadata directory not provided
I20260808 02:21:00.070657 14073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:21:00.075778 14073 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:21:00.079362 14089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:00.081197 14073 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:21:00.081403 14073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "a26e629e3a0343b585265b3da367805b"
format_stamp: "Formatted at 2026-08-08 02:20:57 on dist-test-slave-1vmg"
server_key: "90e9ff9ac4d9d7d1d2bdaaa6af5aeee4"
server_key_iv: "abce51283da1efeeb6ae1b0e1a6b0174"
server_key_version: "encryptionkey@0"
I20260808 02:21:00.082118 14073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:21:00.098825 14073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:21:00.102654 14073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:21:00.102867 14073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:21:00.111316 14073 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:43955
I20260808 02:21:00.111649 14163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:43955 every 8 connection(s)
I20260808 02:21:00.112602 14073 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:21:00.113109 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 14073
I20260808 02:21:00.113603 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:34449
--local_ip_for_outbound_sockets=127.24.79.193
--tserver_master_addrs=127.24.79.254:43955
--webserver_port=34277
--webserver_interface=127.24.79.193
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:46371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:21:00.118243 14164 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:21:00.120385 14164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b: Bootstrap starting.
I20260808 02:21:00.140851 14164 log.cc:824] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b: Log is configured to *not* fsync() on all Append() calls
I20260808 02:21:00.155190 14164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:21:00.155639 14164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b: Bootstrap complete.
I20260808 02:21:00.160164 14164 raft_consensus.cc:348] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26e629e3a0343b585265b3da367805b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 43955 } }
I20260808 02:21:00.160651 14164 raft_consensus.cc:729] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a26e629e3a0343b585265b3da367805b, State: Initialized, Role: FOLLOWER
I20260808 02:21:00.161216 14164 consensus_queue.cc:260] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26e629e3a0343b585265b3da367805b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 43955 } }
I20260808 02:21:00.161358 14164 raft_consensus.cc:388] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:21:00.161430 14164 raft_consensus.cc:482] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:21:00.161540 14164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 2 FOLLOWER]: Advancing to term 3
I20260808 02:21:00.163929 14164 raft_consensus.cc:504] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26e629e3a0343b585265b3da367805b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 43955 } }
I20260808 02:21:00.164348 14164 leader_election.cc:302] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a26e629e3a0343b585265b3da367805b; no voters:
I20260808 02:21:00.164713 14164 leader_election.cc:288] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 02:21:00.164834 14171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 3 FOLLOWER]: Leader election won for term 3
I20260808 02:21:00.165047 14171 raft_consensus.cc:686] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [term 3 LEADER]: Becoming Leader. State: Replica: a26e629e3a0343b585265b3da367805b, State: Running, Role: LEADER
I20260808 02:21:00.165398 14171 consensus_queue.cc:237] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26e629e3a0343b585265b3da367805b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 43955 } }
I20260808 02:21:00.166226 14164 sys_catalog.cc:564] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:21:00.167127 14172 sys_catalog.cc:455] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a26e629e3a0343b585265b3da367805b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26e629e3a0343b585265b3da367805b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 43955 } } }
I20260808 02:21:00.167167 14173 sys_catalog.cc:455] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a26e629e3a0343b585265b3da367805b. Latest consensus state: current_term: 3 leader_uuid: "a26e629e3a0343b585265b3da367805b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26e629e3a0343b585265b3da367805b" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 43955 } } }
I20260808 02:21:00.167315 14173 sys_catalog.cc:458] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [sys.catalog]: This master's current role is: LEADER
I20260808 02:21:00.167313 14172 sys_catalog.cc:458] T 00000000000000000000000000000000 P a26e629e3a0343b585265b3da367805b [sys.catalog]: This master's current role is: LEADER
I20260808 02:21:00.167863 14180 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:21:00.174444 14180 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:21:00.175451 14180 catalog_manager.cc:1259] Loaded cluster ID: 3576271504874753978c9ac6f294582e
I20260808 02:21:00.175555 14180 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 02:21:00.177057 14180 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f85c92869ad google::LogMessage::Fail() at ??:0
@ 0x7f85c928988a google::LogMessage::SendToLog() at ??:0
@ 0x7f85c92864de google::LogMessage::Flush() at ??:0
@ 0x7f85c9287a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f85d3bc9fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f85d3bcad2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f85d3bc7579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f85d3c0b640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f85cb4d83d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f85cb4d8c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f85cb4da3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f85cb4c9512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f85ccb0d6db start_thread at ??:0
@ 0x7f85c7d8461f clone at ??:0
W20260808 02:21:00.264210 14168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:21:00.264515 14168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:21:00.264562 14168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:21:00.269297 14168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:21:00.269371 14168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:21:00.269434 14168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:21:00.275797 14168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:46371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:34449
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=34277
--tserver_master_addrs=127.24.79.254:43955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:21:00.277156 14168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:21:00.278620 14168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:21:00.290493 14219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:21:00.290493 14220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:21:00.291085 14222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:21:00.291301 14168 server_base.cc:1034] running on GCE node
I20260808 02:21:00.291831 14168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:21:00.292752 14168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:21:00.294056 14168 hybrid_clock.cc:648] HybridClock initialized: now 1786155660294023 us; error 57 us; skew 500 ppm
Aug 08 02:21:00 dist-test-slave-1vmg krb5kdc[12549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155660, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:21:00.297571 14168 init.cc:377] Logged in from keytab as kudu/127.24.79.193@KRBTEST.COM (short username kudu)
I20260808 02:21:00.298902 14168 webserver.cc:460] Webserver started at http://127.24.79.193:34277/ using document root <none> and password file <none>
I20260808 02:21:00.299654 14168 fs_manager.cc:359] Metadata directory not provided
I20260808 02:21:00.299743 14168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:21:00.304697 14168 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 02:21:00.307946 14236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:00.309844 14168 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:21:00.310029 14168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "1b385bef71934d1e9c4e8db6120530fb"
format_stamp: "Formatted at 2026-08-08 02:20:57 on dist-test-slave-1vmg"
server_key: "0878b4f99b348c58f2ffffe5cd416dfd"
server_key_iv: "80d778bc23ee007807ea0b2a390cdc63"
server_key_version: "encryptionkey@0"
I20260808 02:21:00.310739 14168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:21:00.325505 14168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:21:00.329298 14168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:21:00.329514 14168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:21:00.330232 14168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:21:00.331606 14245 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 02:21:00.336537 14168 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 02:21:00.336632 14168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260808 02:21:00.336694 14168 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 02:21:00.338660 14168 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 02:21:00.338752 14168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 02:21:00.339108 14245 tablet_bootstrap.cc:492] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb: Bootstrap starting.
I20260808 02:21:00.350952 14168 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:34449
I20260808 02:21:00.351375 14403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:34449 every 8 connection(s)
I20260808 02:21:00.352569 14168 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:21:00.353483 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 14168
I20260808 02:21:00.354031 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:34687
--local_ip_for_outbound_sockets=127.24.79.194
--tserver_master_addrs=127.24.79.254:43955
--webserver_port=35199
--webserver_interface=127.24.79.194
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:46371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:21:00.362725 14245 log.cc:824] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb: Log is configured to *not* fsync() on all Append() calls
I20260808 02:21:00.375511 14245 tablet_bootstrap.cc:492] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:21:00.376062 14245 tablet_bootstrap.cc:492] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb: Bootstrap complete.
I20260808 02:21:00.377796 14245 ts_tablet_manager.cc:1397] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb: Time spent bootstrapping tablet: real 0.039s user 0.024s sys 0.011s
I20260808 02:21:00.381675 14245 raft_consensus.cc:348] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:21:00.382375 14245 raft_consensus.cc:729] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b385bef71934d1e9c4e8db6120530fb, State: Initialized, Role: FOLLOWER
I20260808 02:21:00.383136 14245 consensus_queue.cc:260] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:21:00.384192 14245 ts_tablet_manager.cc:1428] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
W20260808 02:21:00.462081 14406 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:21:00.355804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.193:45483)
0808 02:21:00.358006 (+ 2202us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:00.358088 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0808 02:21:00.359383 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:21:00.460642 (+101259us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1897,"thread_start_us":207,"threads_started":1}
W20260808 02:21:00.463114 14404 heartbeater.cc:644] Failed to heartbeat to 127.24.79.254:43955 (0 consecutive failures): Network error: Failed to ping master at 127.24.79.254:43955: Client connection negotiation failed: client connection to 127.24.79.254:43955: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 02:21:00.464102 14406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:00.463719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.193:52051)
0808 02:21:00.463896 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:00.464005 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 02:21:00.465000 14406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:00.464736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.193:36949)
0808 02:21:00.464862 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:00.464928 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 02:21:00.465342 14404 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:21:00.505970 14407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:21:00.506309 14407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:21:00.506356 14407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:21:00.510655 14407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:21:00.510769 14407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:21:00.510831 14407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:21:00.517611 14407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:46371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:34687
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=35199
--tserver_master_addrs=127.24.79.254:43955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:21:00.518952 14407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:21:00.520141 14407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:21:00.532936 14418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:21:00.532949 14417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:21:00.533248 14407 server_base.cc:1034] running on GCE node
W20260808 02:21:00.533082 14420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:21:00.533869 14407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:21:00.534745 14407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:21:00.536017 14407 hybrid_clock.cc:648] HybridClock initialized: now 1786155660535967 us; error 70 us; skew 500 ppm
Aug 08 02:21:00 dist-test-slave-1vmg krb5kdc[12549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155660, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:21:00.540025 14407 init.cc:377] Logged in from keytab as kudu/127.24.79.194@KRBTEST.COM (short username kudu)
I20260808 02:21:00.541240 14407 webserver.cc:460] Webserver started at http://127.24.79.194:35199/ using document root <none> and password file <none>
I20260808 02:21:00.541917 14407 fs_manager.cc:359] Metadata directory not provided
I20260808 02:21:00.542006 14407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:21:00.547456 14407 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:21:00.550576 14431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:00.552220 14407 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:21:00.552397 14407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "195a12ab501c46058d4d4733c3cf3835"
format_stamp: "Formatted at 2026-08-08 02:20:58 on dist-test-slave-1vmg"
server_key: "8235a3b1041433774ce3788405e123b0"
server_key_iv: "705a8737909dfee1ab0f9ef5cf8ad11d"
server_key_version: "encryptionkey@0"
I20260808 02:21:00.553050 14407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:21:00.577950 14407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:21:00.581929 14407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:21:00.582140 14407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:21:00.582834 14407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:21:00.583953 14407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:21:00.584010 14407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:00.584056 14407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:21:00.584071 14407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:00.595368 14407 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:34687
I20260808 02:21:00.595389 14589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:34687 every 8 connection(s)
I20260808 02:21:00.596539 14407 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:21:00.596944 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 14407
I20260808 02:21:00.597443 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:41277
--local_ip_for_outbound_sockets=127.24.79.195
--tserver_master_addrs=127.24.79.254:43955
--webserver_port=38049
--webserver_interface=127.24.79.195
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:46371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:21:00.602295 14593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:00.599441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.194:44859)
0808 02:21:00.600193 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:00.600399 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":144,"threads_started":1}
W20260808 02:21:00.603008 14590 heartbeater.cc:644] Failed to heartbeat to 127.24.79.254:43955 (0 consecutive failures): Network error: Failed to ping master at 127.24.79.254:43955: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
I20260808 02:21:00.604043 14593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:00.603600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.194:36963)
0808 02:21:00.603858 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:00.603947 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 02:21:00.605101 14593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:00.604807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.194:36417)
0808 02:21:00.604948 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:00.605013 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 02:21:00.605367 14590 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:21:00.733613 14592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:21:00.733954 14592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:21:00.734002 14592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:21:00.738181 14592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:21:00.738257 14592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:21:00.738319 14592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:21:00.744268 14592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:46371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:41277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=38049
--tserver_master_addrs=127.24.79.254:43955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:21:00.745537 14592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:21:00.746650 14592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:21:00.757740 14599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:21:00.757841 14602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:21:00.757880 14592 server_base.cc:1034] running on GCE node
W20260808 02:21:00.757735 14600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:21:00.758608 14592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:21:00.759605 14592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:21:00.760823 14592 hybrid_clock.cc:648] HybridClock initialized: now 1786155660760799 us; error 40 us; skew 500 ppm
Aug 08 02:21:00 dist-test-slave-1vmg krb5kdc[12549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155660, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:21:00.764740 14592 init.cc:377] Logged in from keytab as kudu/127.24.79.195@KRBTEST.COM (short username kudu)
I20260808 02:21:00.766054 14592 webserver.cc:460] Webserver started at http://127.24.79.195:38049/ using document root <none> and password file <none>
I20260808 02:21:00.766772 14592 fs_manager.cc:359] Metadata directory not provided
I20260808 02:21:00.766870 14592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:21:00.772001 14592 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:21:00.775511 14613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:00.777542 14592 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:21:00.777736 14592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "204278d3b4cb446c932dba11b4dff15e"
format_stamp: "Formatted at 2026-08-08 02:20:58 on dist-test-slave-1vmg"
server_key: "176f6b683a4898042d84e3d4e91a9991"
server_key_iv: "6b63bf09a10b11ff9ceb5497ef5f5809"
server_key_version: "encryptionkey@0"
I20260808 02:21:00.778388 14592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:21:00.798578 14592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:21:00.802435 14592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:21:00.802699 14592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:21:00.803522 14592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:21:00.804558 14592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:21:00.804620 14592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:00.804683 14592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:21:00.804714 14592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:00.815171 14592 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:41277
I20260808 02:21:00.815311 14775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:41277 every 8 connection(s)
I20260808 02:21:00.816623 14592 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:21:00.818643 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 14592
I20260808 02:21:00.821918 14779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:00.819629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.195:49805)
0808 02:21:00.820290 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:00.820443 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":234,"threads_started":1}
W20260808 02:21:00.822535 14776 heartbeater.cc:644] Failed to heartbeat to 127.24.79.254:43955 (0 consecutive failures): Network error: Failed to ping master at 127.24.79.254:43955: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
I20260808 02:21:00.823349 14779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:00.823091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.195:48381)
0808 02:21:00.823221 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:00.823280 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 02:21:00.824146 14779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:00.823891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.195:46453)
0808 02:21:00.824030 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:00.824083 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260808 02:21:00.824399 14776 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 02:21:01.466887 14780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:01.466364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.193:39585)
0808 02:21:01.466697 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:01.466758 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":105,"threads_started":1}
I20260808 02:21:01.606894 14781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:01.606304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.194:53647)
0808 02:21:01.606680 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:01.606763 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":120,"threads_started":1}
I20260808 02:21:01.818539 14782 raft_consensus.cc:482] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:21:01.818794 14782 raft_consensus.cc:504] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:21:01.820372 14782 leader_election.cc:288] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 204278d3b4cb446c932dba11b4dff15e (127.24.79.195:41277), 195a12ab501c46058d4d4733c3cf3835 (127.24.79.194:34687)
I20260808 02:21:01.825547 14786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:01.825043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.195:37503)
0808 02:21:01.825342 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:01.825409 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":115,"threads_started":1}
Aug 08 02:21:01 dist-test-slave-1vmg krb5kdc[12549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155660, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.194@KRBTEST.COM
Aug 08 02:21:01 dist-test-slave-1vmg krb5kdc[12549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155660, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.195@KRBTEST.COM
I20260808 02:21:01.841305 14783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:01.820687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:34687 (local address 127.24.79.193:56265)
0808 02:21:01.821089 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:01.821103 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:21:01.821210 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:21:01.822951 (+ 1741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:21:01.822959 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:21:01.823569 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:21:01.824151 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:01.824172 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:01.825749 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:01.825752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:01.826901 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:01.826941 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:01.827737 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:01.827741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:01.827974 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:01.833005 (+ 5031us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:21:01.833031 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:21:01.835159 (+ 2128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:21:01.838321 (+ 3162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:21:01.838324 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:21:01.838335 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:21:01.838804 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:21:01.839289 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:21:01.839293 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:21:01.839294 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:21:01.839471 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:21:01.840248 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:21:01.840256 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:21:01.840594 (+ 338us) client_negotiation.cc:769] Sending connection context
0808 02:21:01.840768 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 02:21:01.841131 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":291,"spinlock_wait_cycles":27392,"thread_start_us":98,"threads_started":1}
I20260808 02:21:01.841807 14780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:01.820684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:41277 (local address 127.24.79.193:37763)
0808 02:21:01.820893 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:01.820908 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:21:01.821021 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:21:01.822965 (+ 1944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:21:01.822969 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:21:01.823575 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:21:01.824151 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:01.824171 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:01.825745 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:01.825748 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:01.826900 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:01.826938 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:01.827737 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:01.827741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:01.827984 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:01.833007 (+ 5023us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:21:01.833041 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:21:01.835540 (+ 2499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:21:01.838310 (+ 2770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:21:01.838318 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:21:01.838333 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:21:01.838808 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:21:01.839307 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:21:01.839311 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:21:01.839312 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:21:01.839477 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:21:01.840252 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:21:01.840259 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:21:01.840577 (+ 318us) client_negotiation.cc:769] Sending connection context
0808 02:21:01.840768 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 02:21:01.841637 (+ 869us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":223,"spinlock_wait_cycles":57728}
I20260808 02:21:01.841872 14785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:01.821078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:37763 (local address 127.24.79.195:41277)
0808 02:21:01.821397 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:01.821408 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:01.821440 (+ 32us) server_negotiation.cc:407] Connection header received
0808 02:21:01.822082 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:01.822089 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:01.822491 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:01.822806 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:21:01.824356 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:01.825552 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:01.827072 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:01.827592 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:01.827758 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:01.835829 (+ 8071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:21:01.835838 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:21:01.835855 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:21:01.835903 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:21:01.838092 (+ 2189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:01.839028 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:01.839035 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:01.839040 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:01.839195 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:01.839601 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:01.839604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:01.839605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:01.840116 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:21:01.840267 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 02:21:01.840960 (+ 693us) server_negotiation.cc:299] Negotiation successful
0808 02:21:01.841305 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":119,"threads_started":1}
I20260808 02:21:01.842372 14784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:01.820944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:56265 (local address 127.24.79.194:34687)
0808 02:21:01.821298 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:01.821327 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:01.821376 (+ 49us) server_negotiation.cc:407] Connection header received
0808 02:21:01.822083 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:01.822090 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:01.822477 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:01.822808 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:21:01.824358 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:01.825562 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:01.827059 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:01.827595 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:01.827760 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:01.835505 (+ 7745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:21:01.835514 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:21:01.835531 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:21:01.835573 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:21:01.838092 (+ 2519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:01.839004 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:01.839011 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:01.839016 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:01.839171 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:01.839583 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:01.839586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:01.839588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:01.840115 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:21:01.840267 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 02:21:01.841839 (+ 1572us) server_negotiation.cc:299] Negotiation successful
0808 02:21:01.842186 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":71,"threads_started":1}
W20260808 02:21:01.845009 14241 leader_election.cc:341] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 195a12ab501c46058d4d4733c3cf3835 (127.24.79.194:34687): Not found: Tablet not found: e7fc511944f04f748270034c95b9ae45
W20260808 02:21:01.845252 14239 leader_election.cc:341] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 204278d3b4cb446c932dba11b4dff15e (127.24.79.195:41277): Not found: Tablet not found: e7fc511944f04f748270034c95b9ae45
I20260808 02:21:01.845341 14239 leader_election.cc:302] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b385bef71934d1e9c4e8db6120530fb; no voters: 195a12ab501c46058d4d4733c3cf3835, 204278d3b4cb446c932dba11b4dff15e
I20260808 02:21:01.845624 14782 raft_consensus.cc:2726] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:21:02.468631 14787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:02.467975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.193:34299)
0808 02:21:02.468430 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:02.468517 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":91,"threads_started":1}
I20260808 02:21:02.608630 14788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:02.608041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.194:40511)
0808 02:21:02.608444 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:02.608520 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":92,"threads_started":1}
I20260808 02:21:02.827206 14789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:02.826631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.195:53277)
0808 02:21:02.827022 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:02.827094 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":89,"threads_started":1}
I20260808 02:21:03.470346 14790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:03.469790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.193:54613)
0808 02:21:03.470145 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:03.470218 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":97,"threads_started":1}
I20260808 02:21:03.610350 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:03.609820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.194:33075)
0808 02:21:03.610109 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:03.610181 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":91,"threads_started":1}
I20260808 02:21:03.815924 14796 raft_consensus.cc:482] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:21:03.816108 14796 raft_consensus.cc:504] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:21:03.816684 14796 leader_election.cc:288] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 204278d3b4cb446c932dba11b4dff15e (127.24.79.195:41277), 195a12ab501c46058d4d4733c3cf3835 (127.24.79.194:34687)
W20260808 02:21:03.817706 14241 leader_election.cc:341] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 195a12ab501c46058d4d4733c3cf3835 (127.24.79.194:34687): Not found: Tablet not found: e7fc511944f04f748270034c95b9ae45
W20260808 02:21:03.817886 14239 leader_election.cc:341] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 204278d3b4cb446c932dba11b4dff15e (127.24.79.195:41277): Not found: Tablet not found: e7fc511944f04f748270034c95b9ae45
I20260808 02:21:03.817984 14239 leader_election.cc:302] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b385bef71934d1e9c4e8db6120530fb; no voters: 195a12ab501c46058d4d4733c3cf3835, 204278d3b4cb446c932dba11b4dff15e
I20260808 02:21:03.818147 14796 raft_consensus.cc:2726] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:21:03.828845 14797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:03.828210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.195:60927)
0808 02:21:03.828655 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:03.828735 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":182,"threads_started":1}
I20260808 02:21:04.472023 14799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:04.471487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.193:45021)
0808 02:21:04.471828 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:04.471912 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":95,"threads_started":1}
I20260808 02:21:04.612272 14800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:04.611587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.194:47075)
0808 02:21:04.611981 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:04.612057 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":91,"threads_started":1}
I20260808 02:21:04.830195 14801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:04.829751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.195:39177)
0808 02:21:04.830011 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:04.830071 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":83,"threads_started":1}
I20260808 02:21:05.474056 14802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:05.473350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.193:38203)
0808 02:21:05.473828 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:05.473922 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":184,"threads_started":1}
I20260808 02:21:05.506196 14803 raft_consensus.cc:482] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:21:05.506361 14803 raft_consensus.cc:504] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:21:05.506891 14803 leader_election.cc:288] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 204278d3b4cb446c932dba11b4dff15e (127.24.79.195:41277), 195a12ab501c46058d4d4733c3cf3835 (127.24.79.194:34687)
W20260808 02:21:05.507843 14241 leader_election.cc:341] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 195a12ab501c46058d4d4733c3cf3835 (127.24.79.194:34687): Not found: Tablet not found: e7fc511944f04f748270034c95b9ae45
W20260808 02:21:05.508073 14239 leader_election.cc:341] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 204278d3b4cb446c932dba11b4dff15e (127.24.79.195:41277): Not found: Tablet not found: e7fc511944f04f748270034c95b9ae45
I20260808 02:21:05.508285 14239 leader_election.cc:302] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b385bef71934d1e9c4e8db6120530fb; no voters: 195a12ab501c46058d4d4733c3cf3835, 204278d3b4cb446c932dba11b4dff15e
I20260808 02:21:05.508559 14803 raft_consensus.cc:2726] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:21:05.613883 14804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:05.613260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.194:37725)
0808 02:21:05.613681 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:05.613757 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":122,"threads_started":1}
I20260808 02:21:05.831952 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:05.831296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.195:53897)
0808 02:21:05.831663 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:05.831746 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":110,"threads_started":1}
I20260808 02:21:06.475689 14809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:06.475173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.193:36479)
0808 02:21:06.475494 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:06.475570 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":86,"threads_started":1}
I20260808 02:21:06.615679 14810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:06.615037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.194:45887)
0808 02:21:06.615446 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:06.615545 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":144,"threads_started":1}
I20260808 02:21:06.833431 14814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:06.832922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.195:43379)
0808 02:21:06.833253 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:06.833322 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":86,"threads_started":1}
I20260808 02:21:07.477391 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:07.476764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.193:39947)
0808 02:21:07.477195 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:07.477279 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":156,"threads_started":1}
I20260808 02:21:07.617362 14816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:07.616764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.194:46023)
0808 02:21:07.617125 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:07.617242 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":85,"threads_started":1}
I20260808 02:21:07.835295 14818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:07.834600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.195:39345)
0808 02:21:07.835100 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:07.835181 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":120,"threads_started":1}
I20260808 02:21:08.478812 14819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:08.478266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.193:42395)
0808 02:21:08.478623 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:08.478703 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":86,"threads_started":1}
I20260808 02:21:08.618957 14820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:08.618386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.194:55893)
0808 02:21:08.618733 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:08.618809 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":101,"threads_started":1}
I20260808 02:21:08.837056 14821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:08.836483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.195:42775)
0808 02:21:08.836849 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:08.836925 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":102,"threads_started":1}
I20260808 02:21:09.480247 14822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:09.479748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.193:36029)
0808 02:21:09.480047 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:09.480139 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":85,"threads_started":1}
I20260808 02:21:09.620421 14823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:09.619865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.194:53777)
0808 02:21:09.620172 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:09.620301 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":83,"threads_started":1}
I20260808 02:21:09.838699 14825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:09.838124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.195:43789)
0808 02:21:09.838494 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:09.838579 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":92,"threads_started":1}
I20260808 02:21:09.929735 14826 raft_consensus.cc:482] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:21:09.929884 14826 raft_consensus.cc:504] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:21:09.930384 14826 leader_election.cc:288] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 204278d3b4cb446c932dba11b4dff15e (127.24.79.195:41277), 195a12ab501c46058d4d4733c3cf3835 (127.24.79.194:34687)
W20260808 02:21:09.931293 14239 leader_election.cc:341] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 204278d3b4cb446c932dba11b4dff15e (127.24.79.195:41277): Not found: Tablet not found: e7fc511944f04f748270034c95b9ae45
W20260808 02:21:09.931423 14241 leader_election.cc:341] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 195a12ab501c46058d4d4733c3cf3835 (127.24.79.194:34687): Not found: Tablet not found: e7fc511944f04f748270034c95b9ae45
I20260808 02:21:09.931525 14241 leader_election.cc:302] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b385bef71934d1e9c4e8db6120530fb; no voters: 195a12ab501c46058d4d4733c3cf3835, 204278d3b4cb446c932dba11b4dff15e
I20260808 02:21:09.931716 14826 raft_consensus.cc:2726] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:21:10.481775 14827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:10.481289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.193:35065)
0808 02:21:10.481593 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:10.481667 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":107,"threads_started":1}
I20260808 02:21:10.622057 14828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:10.621517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.194:35629)
0808 02:21:10.621857 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:10.621938 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":88,"threads_started":1}
I20260808 02:21:10.840389 14829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:10.839862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.195:59523)
0808 02:21:10.840182 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:10.840259 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":94,"threads_started":1}
I20260808 02:21:11.483429 14830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:11.482802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.193:54607)
0808 02:21:11.483197 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:11.483297 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":127,"threads_started":1}
I20260808 02:21:11.623854 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:11.623304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.194:56561)
0808 02:21:11.623650 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:11.623732 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":91,"threads_started":1}
I20260808 02:21:11.841691 14834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:11.841215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.195:40515)
0808 02:21:11.841516 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:11.841580 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":86,"threads_started":1}
I20260808 02:21:12.485144 14836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:12.484457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.193:44341)
0808 02:21:12.484906 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:12.484977 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":155,"threads_started":1}
I20260808 02:21:12.625257 14837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:12.624711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.194:52799)
0808 02:21:12.625064 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:12.625131 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":92,"threads_started":1}
I20260808 02:21:12.843436 14842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:12.842851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.195:33187)
0808 02:21:12.843241 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:12.843319 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":110,"threads_started":1}
I20260808 02:21:13.486903 14843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:13.486358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.193:58899)
0808 02:21:13.486703 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:13.486783 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":93,"threads_started":1}
I20260808 02:21:13.596316 14844 raft_consensus.cc:482] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:21:13.596487 14844 raft_consensus.cc:504] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "204278d3b4cb446c932dba11b4dff15e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 41277 } } peers { permanent_uuid: "195a12ab501c46058d4d4733c3cf3835" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 34687 } } peers { permanent_uuid: "1b385bef71934d1e9c4e8db6120530fb" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 34449 } }
I20260808 02:21:13.597258 14844 leader_election.cc:288] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 204278d3b4cb446c932dba11b4dff15e (127.24.79.195:41277), 195a12ab501c46058d4d4733c3cf3835 (127.24.79.194:34687)
W20260808 02:21:13.598188 14241 leader_election.cc:341] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 195a12ab501c46058d4d4733c3cf3835 (127.24.79.194:34687): Not found: Tablet not found: e7fc511944f04f748270034c95b9ae45
W20260808 02:21:13.598467 14239 leader_election.cc:341] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 204278d3b4cb446c932dba11b4dff15e (127.24.79.195:41277): Not found: Tablet not found: e7fc511944f04f748270034c95b9ae45
I20260808 02:21:13.598583 14239 leader_election.cc:302] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b385bef71934d1e9c4e8db6120530fb; no voters: 195a12ab501c46058d4d4733c3cf3835, 204278d3b4cb446c932dba11b4dff15e
I20260808 02:21:13.598800 14844 raft_consensus.cc:2726] T e7fc511944f04f748270034c95b9ae45 P 1b385bef71934d1e9c4e8db6120530fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:21:13.627072 14846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:13.626484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.194:57515)
0808 02:21:13.626843 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:13.626945 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":108,"threads_started":1}
I20260808 02:21:13.845208 14847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:13.844631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.195:39091)
0808 02:21:13.845021 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:13.845095 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":98,"threads_started":1}
I20260808 02:21:14.488477 14848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:14.487853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.193:50363)
0808 02:21:14.488200 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:14.488364 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":91,"threads_started":1}
I20260808 02:21:14.628933 14850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:14.628236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.194:42079)
0808 02:21:14.628626 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:14.628814 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":91,"threads_started":1}
I20260808 02:21:14.846870 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:14.846339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.195:47511)
0808 02:21:14.846682 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:14.846757 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":93,"threads_started":1}
I20260808 02:21:15.490594 14853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:15.489966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.193:40733)
0808 02:21:15.490318 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:15.490471 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":90,"threads_started":1}
I20260808 02:21:15.630885 14855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:15.630313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.194:50883)
0808 02:21:15.630706 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:15.630780 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":95,"threads_started":1}
I20260808 02:21:15.832069 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 14168
I20260808 02:21:15.848887 14857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:15.848332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:43955 (local address 127.24.79.195:33937)
0808 02:21:15.848696 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:15.848775 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":106,"threads_started":1}
I20260808 02:21:15.854104 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 14407
I20260808 02:21:15.885495 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 14592
2026-08-08T02:21:15Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (20410 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:21:16 dist-test-slave-1vmg krb5kdc[14862](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:21:16 dist-test-slave-1vmg krb5kdc[14862](info): set up 2 sockets
Aug 08 02:21:16 dist-test-slave-1vmg krb5kdc[14862](info): commencing operation
krb5kdc: starting...
W20260808 02:21:17.516511 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:21:17 dist-test-slave-1vmg krb5kdc[14862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:21:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:21:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:21:17.675529 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:38659
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:40593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:38659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:21:17.811847 14879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:21:17.812282 14879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:21:17.812331 14879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:21:17.816743 14879 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 02:21:17.816953 14879 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 02:21:17.816993 14879 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 02:21:17.817008 14879 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:21:17.817030 14879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:21:17.817055 14879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:21:17.817075 14879 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:21:17.817097 14879 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:21:17.824019 14879 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:38659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:38659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:21:17.825327 14879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:21:17.826510 14879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:21:17.837008 14888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:21:17.837267 14879 server_base.cc:1034] running on GCE node
W20260808 02:21:17.837050 14893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:21:17.837050 14887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:21:17.838160 14879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:21:17.839603 14879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:21:17.840842 14879 hybrid_clock.cc:648] HybridClock initialized: now 1786155677840819 us; error 44 us; skew 500 ppm
Aug 08 02:21:17 dist-test-slave-1vmg krb5kdc[14862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155677, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:21:17.844372 14879 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:21:17.845549 14879 webserver.cc:460] Webserver started at http://127.24.79.254:42817/ using document root <none> and password file <none>
I20260808 02:21:17.846333 14879 fs_manager.cc:359] Metadata directory not provided
I20260808 02:21:17.846457 14879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:21:17.846736 14879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:21:17.849308 14879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "69b642c340c5439dafce5fb8caefdce0"
format_stamp: "Formatted at 2026-08-08 02:21:17 on dist-test-slave-1vmg"
server_key: "67108216d134b7ee5cfcd1d43a781919"
server_key_iv: "b0ea4e4b14fb21ecbbe9305acb627156"
server_key_version: "encryptionkey@0"
I20260808 02:21:17.850005 14879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "69b642c340c5439dafce5fb8caefdce0"
format_stamp: "Formatted at 2026-08-08 02:21:17 on dist-test-slave-1vmg"
server_key: "67108216d134b7ee5cfcd1d43a781919"
server_key_iv: "b0ea4e4b14fb21ecbbe9305acb627156"
server_key_version: "encryptionkey@0"
I20260808 02:21:17.854435 14879 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260808 02:21:17.857723 14904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:17.859467 14879 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:21:17.859639 14879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "69b642c340c5439dafce5fb8caefdce0"
format_stamp: "Formatted at 2026-08-08 02:21:17 on dist-test-slave-1vmg"
server_key: "67108216d134b7ee5cfcd1d43a781919"
server_key_iv: "b0ea4e4b14fb21ecbbe9305acb627156"
server_key_version: "encryptionkey@0"
I20260808 02:21:17.859797 14879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:21:17.880482 14879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:21:17.884608 14879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:21:17.884846 14879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:21:17.893059 14879 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:38659
I20260808 02:21:17.893435 14974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:38659 every 8 connection(s)
I20260808 02:21:17.894218 14879 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:21:17.895490 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 14879
I20260808 02:21:17.895663 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:21:17.895967 24895 external_mini_cluster.cc:1428] Setting key 4d3aa83cfb1e9dc476d6fbfe10523333
I20260808 02:21:17.899008 14975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 02:21:17 dist-test-slave-1vmg krb5kdc[14862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:21:17.906131 14975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0: Bootstrap starting.
I20260808 02:21:17.909427 14975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0: Neither blocks nor log segments found. Creating new log.
I20260808 02:21:17.910710 14975 log.cc:824] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0: Log is configured to *not* fsync() on all Append() calls
I20260808 02:21:17.912738 14980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:17.897435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47214 (local address 127.24.79.254:38659)
0808 02:21:17.897952 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:17.897970 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:17.898013 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:21:17.898787 (+ 774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:17.898807 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:17.899286 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:17.899665 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:21:17.900596 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:17.901667 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:17.902585 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:17.903183 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:17.903376 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:17.906561 (+ 3185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:21:17.906571 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:21:17.906594 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:21:17.906641 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:21:17.909056 (+ 2415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:17.909794 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:17.909802 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:17.909808 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:17.909997 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:17.910386 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:17.910390 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:17.910391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:17.910970 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:21:17.911174 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 02:21:17.911537 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 02:21:17.911897 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":164,"threads_started":1}
I20260808 02:21:17.913514 14975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0: No bootstrap required, opened a new log
I20260808 02:21:17.917798 14975 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b642c340c5439dafce5fb8caefdce0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38659 } }
I20260808 02:21:17.918111 14975 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:21:17.918159 14975 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69b642c340c5439dafce5fb8caefdce0, State: Initialized, Role: FOLLOWER
I20260808 02:21:17.918658 14975 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b642c340c5439dafce5fb8caefdce0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38659 } }
I20260808 02:21:17.918799 14975 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:21:17.918861 14975 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:21:17.919004 14975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:21:17.920394 14975 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b642c340c5439dafce5fb8caefdce0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38659 } }
I20260808 02:21:17.920789 14975 leader_election.cc:302] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69b642c340c5439dafce5fb8caefdce0; no voters:
I20260808 02:21:17.921167 14975 leader_election.cc:288] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:21:17.921444 14982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:21:17.921722 14982 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 1 LEADER]: Becoming Leader. State: Replica: 69b642c340c5439dafce5fb8caefdce0, State: Running, Role: LEADER
I20260808 02:21:17.922129 14982 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b642c340c5439dafce5fb8caefdce0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38659 } }
I20260808 02:21:17.922545 14975 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:21:17.923844 14985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69b642c340c5439dafce5fb8caefdce0. Latest consensus state: current_term: 1 leader_uuid: "69b642c340c5439dafce5fb8caefdce0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b642c340c5439dafce5fb8caefdce0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38659 } } }
I20260808 02:21:17.924000 14985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [sys.catalog]: This master's current role is: LEADER
I20260808 02:21:17.924471 14984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69b642c340c5439dafce5fb8caefdce0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b642c340c5439dafce5fb8caefdce0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38659 } } }
I20260808 02:21:17.924630 14984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [sys.catalog]: This master's current role is: LEADER
I20260808 02:21:17.925748 14995 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:21:17.928830 14995 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:21:17.935541 14995 catalog_manager.cc:1347] Generated new cluster ID: 071d8474e0dd413bb7ec01c0edfdb77b
I20260808 02:21:17.935664 14995 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:21:17.944154 14995 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:21:17.945606 14995 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:21:17.955574 14995 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0: Generated new TSK 0
I20260808 02:21:17.956676 14995 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.79.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
I20260808 02:21:17.999244 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:38659
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:40593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:21:18.134276 15010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:21:18.134684 15010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:21:18.134770 15010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:21:18.139600 15010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:21:18.139700 15010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:21:18.139818 15010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:21:18.146479 15010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:38659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:21:18.147897 15010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:21:18.149387 15010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:21:18.162315 15016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:21:18.162304 15019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:21:18.162358 15015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:21:18.162972 15010 server_base.cc:1034] running on GCE node
I20260808 02:21:18.163532 15010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:21:18.164906 15010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:21:18.166160 15010 hybrid_clock.cc:648] HybridClock initialized: now 1786155678166136 us; error 45 us; skew 500 ppm
Aug 08 02:21:18 dist-test-slave-1vmg krb5kdc[14862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155678, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:21:18.170001 15010 init.cc:377] Logged in from keytab as kudu/127.24.79.193@KRBTEST.COM (short username kudu)
I20260808 02:21:18.171305 15010 webserver.cc:460] Webserver started at http://127.24.79.193:36793/ using document root <none> and password file <none>
I20260808 02:21:18.171989 15010 fs_manager.cc:359] Metadata directory not provided
I20260808 02:21:18.172075 15010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:21:18.172307 15010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:21:18.174732 15010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "df902d8757214a8da739a4da5255e2e2"
format_stamp: "Formatted at 2026-08-08 02:21:18 on dist-test-slave-1vmg"
server_key: "ddf9554af2f50e833f96134e1208266e"
server_key_iv: "8f1234e252ef67825aacaa24fcb4551f"
server_key_version: "encryptionkey@0"
I20260808 02:21:18.175691 15010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "df902d8757214a8da739a4da5255e2e2"
format_stamp: "Formatted at 2026-08-08 02:21:18 on dist-test-slave-1vmg"
server_key: "ddf9554af2f50e833f96134e1208266e"
server_key_iv: "8f1234e252ef67825aacaa24fcb4551f"
server_key_version: "encryptionkey@0"
I20260808 02:21:18.180343 15010 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260808 02:21:18.183571 15028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:18.185235 15010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:21:18.185384 15010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "df902d8757214a8da739a4da5255e2e2"
format_stamp: "Formatted at 2026-08-08 02:21:18 on dist-test-slave-1vmg"
server_key: "ddf9554af2f50e833f96134e1208266e"
server_key_iv: "8f1234e252ef67825aacaa24fcb4551f"
server_key_version: "encryptionkey@0"
I20260808 02:21:18.185534 15010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:21:18.207770 15010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:21:18.212138 15010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:21:18.212366 15010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:21:18.213133 15010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:21:18.214187 15010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:21:18.214251 15010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:18.214298 15010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:21:18.214313 15010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:18.224620 15010 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:41277
I20260808 02:21:18.224696 15175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:41277 every 8 connection(s)
I20260808 02:21:18.225837 15010 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:21:18.229902 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 15010
I20260808 02:21:18.230069 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:21:18.230564 24895 external_mini_cluster.cc:1428] Setting key f7d37f60d8df24a915bc396438220c44
Aug 08 02:21:18 dist-test-slave-1vmg krb5kdc[14862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155678, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.79.194@KRBTEST.COM; defaulting to no policy
I20260808 02:21:18.244086 14980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:18.228408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:44231 (local address 127.24.79.254:38659)
0808 02:21:18.228606 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:18.228612 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:18.229887 (+ 1275us) server_negotiation.cc:407] Connection header received
0808 02:21:18.231566 (+ 1679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:18.231570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:18.231643 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:18.231721 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:21:18.233417 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.233978 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.235819 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.236086 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.236206 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:18.239115 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:21:18.239119 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:21:18.239121 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:21:18.239156 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:21:18.241066 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:18.241926 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:18.241930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:18.241931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:18.242047 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:18.242602 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:18.242606 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:18.242607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:18.242887 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:21:18.243023 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 02:21:18.243768 (+ 745us) server_negotiation.cc:299] Negotiation successful
0808 02:21:18.243938 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 02:21:18.245019 15178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:18.228955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.193:44231)
0808 02:21:18.229623 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:18.229681 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 02:21:18.231303 (+ 1622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:21:18.231962 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:21:18.231970 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:21:18.232556 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:21:18.233205 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:18.233222 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.234128 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.234132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:18.235520 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:18.235538 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.236300 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.236304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:18.236480 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:18.237133 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:21:18.237154 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:21:18.238840 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:21:18.241253 (+ 2413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:21:18.241264 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:21:18.241291 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:21:18.241747 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:21:18.242168 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:21:18.242181 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:21:18.242183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:21:18.242434 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:21:18.243056 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:21:18.243064 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:21:18.243408 (+ 344us) client_negotiation.cc:769] Sending connection context
0808 02:21:18.243607 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 02:21:18.243980 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":160,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.194@KRBTEST.COM" created.
I20260808 02:21:18.247164 15176 heartbeater.cc:347] Connected to a master server at 127.24.79.254:38659
I20260808 02:21:18.247504 15176 heartbeater.cc:464] Registering TS with master...
I20260808 02:21:18.248256 15176 heartbeater.cc:511] Master 127.24.79.254:38659 requested a full tablet report, sending...
I20260808 02:21:18.250653 14929 ts_manager.cc:194] Registered new tserver with Master: df902d8757214a8da739a4da5255e2e2 (127.24.79.193:41277)
I20260808 02:21:18.251957 14929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.193@KRBTEST.COM'} at 127.24.79.193:44231
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
I20260808 02:21:18.269296 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:38659
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:40593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:21:18.395252 15183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:21:18.395601 15183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:21:18.395668 15183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:21:18.400271 15183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:21:18.400565 15183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:21:18.400708 15183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:21:18.407058 15183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:38659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:21:18.408461 15183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:21:18.409796 15183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:21:18.420926 15183 server_base.cc:1034] running on GCE node
W20260808 02:21:18.420884 15192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:21:18.420884 15196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:21:18.420884 15193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:21:18.421680 15183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:21:18.422462 15183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:21:18.423661 15183 hybrid_clock.cc:648] HybridClock initialized: now 1786155678423595 us; error 69 us; skew 500 ppm
Aug 08 02:21:18 dist-test-slave-1vmg krb5kdc[14862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155678, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:21:18.427220 15183 init.cc:377] Logged in from keytab as kudu/127.24.79.194@KRBTEST.COM (short username kudu)
I20260808 02:21:18.428345 15183 webserver.cc:460] Webserver started at http://127.24.79.194:38163/ using document root <none> and password file <none>
I20260808 02:21:18.428992 15183 fs_manager.cc:359] Metadata directory not provided
I20260808 02:21:18.429077 15183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:21:18.429358 15183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:21:18.431777 15183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "eef8385f4f864c02a4e467439d492ef1"
format_stamp: "Formatted at 2026-08-08 02:21:18 on dist-test-slave-1vmg"
server_key: "0983995ffa15abd31666070d87bbab2e"
server_key_iv: "c2119a443dc760383193fba777d0f6d4"
server_key_version: "encryptionkey@0"
I20260808 02:21:18.432415 15183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "eef8385f4f864c02a4e467439d492ef1"
format_stamp: "Formatted at 2026-08-08 02:21:18 on dist-test-slave-1vmg"
server_key: "0983995ffa15abd31666070d87bbab2e"
server_key_iv: "c2119a443dc760383193fba777d0f6d4"
server_key_version: "encryptionkey@0"
I20260808 02:21:18.436847 15183 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 02:21:18.439872 15204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:18.441586 15183 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:21:18.441757 15183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "eef8385f4f864c02a4e467439d492ef1"
format_stamp: "Formatted at 2026-08-08 02:21:18 on dist-test-slave-1vmg"
server_key: "0983995ffa15abd31666070d87bbab2e"
server_key_iv: "c2119a443dc760383193fba777d0f6d4"
server_key_version: "encryptionkey@0"
I20260808 02:21:18.441900 15183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:21:18.454797 15183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:21:18.458520 15183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:21:18.458730 15183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:21:18.459474 15183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:21:18.460441 15183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:21:18.460496 15183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:18.460554 15183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:21:18.460579 15183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:18.470314 15183 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:44439
I20260808 02:21:18.470340 15364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:44439 every 8 connection(s)
I20260808 02:21:18.471431 15183 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:21:18.478858 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 15183
I20260808 02:21:18.479017 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:21:18.479310 24895 external_mini_cluster.cc:1428] Setting key 23a9b375d03f81f93c4c2d27ad918104
Aug 08 02:21:18 dist-test-slave-1vmg krb5kdc[14862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155678, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:21:18.486997 14980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:18.473678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:50623 (local address 127.24.79.254:38659)
0808 02:21:18.473866 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:18.473869 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:18.475101 (+ 1232us) server_negotiation.cc:407] Connection header received
0808 02:21:18.476058 (+ 957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:18.476062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:18.476125 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:18.476206 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:21:18.477562 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.478111 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.479433 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.479757 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.479880 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:18.482799 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:21:18.482804 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:21:18.482807 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:21:18.482840 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:21:18.484526 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:18.485130 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:18.485135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:18.485137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:18.485255 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:18.485654 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:18.485657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:18.485659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:18.485907 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:21:18.486010 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 02:21:18.486624 (+ 614us) server_negotiation.cc:299] Negotiation successful
0808 02:21:18.486784 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 02:21:18.487927 15369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:18.474120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.194:50623)
0808 02:21:18.474866 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:18.474960 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0808 02:21:18.475827 (+ 867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:21:18.476453 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:21:18.476461 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:21:18.476865 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:21:18.477401 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:18.477416 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.478248 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.478250 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:18.479288 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:18.479300 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.479958 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.479962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:18.480136 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:18.480945 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:21:18.480968 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:21:18.482593 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:21:18.484670 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:21:18.484678 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:21:18.484695 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:21:18.485009 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:21:18.485367 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:21:18.485370 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:21:18.485372 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:21:18.485550 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:21:18.486006 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:21:18.486012 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:21:18.486349 (+ 337us) client_negotiation.cc:769] Sending connection context
0808 02:21:18.486538 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 02:21:18.486899 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":129,"threads_started":1}
I20260808 02:21:18.489202 15366 heartbeater.cc:347] Connected to a master server at 127.24.79.254:38659
I20260808 02:21:18.489468 15366 heartbeater.cc:464] Registering TS with master...
I20260808 02:21:18.490008 15366 heartbeater.cc:511] Master 127.24.79.254:38659 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.24.79.195@KRBTEST.COM; defaulting to no policy
I20260808 02:21:18.491223 14929 ts_manager.cc:194] Registered new tserver with Master: eef8385f4f864c02a4e467439d492ef1 (127.24.79.194:44439)
I20260808 02:21:18.491982 14929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.194@KRBTEST.COM'} at 127.24.79.194:50623
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
I20260808 02:21:18.515586 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:38659
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:40593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:21:18.645386 15373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:21:18.645725 15373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:21:18.645773 15373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:21:18.650445 15373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:21:18.650555 15373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:21:18.650629 15373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:21:18.656720 15373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:38659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:21:18.658116 15373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:21:18.659408 15373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:21:18.671416 15378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:21:18.671487 15373 server_base.cc:1034] running on GCE node
W20260808 02:21:18.671417 15379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:21:18.671438 15381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:21:18.672180 15373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:21:18.672927 15373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:21:18.674127 15373 hybrid_clock.cc:648] HybridClock initialized: now 1786155678674077 us; error 68 us; skew 500 ppm
Aug 08 02:21:18 dist-test-slave-1vmg krb5kdc[14862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155678, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:21:18.677348 15373 init.cc:377] Logged in from keytab as kudu/127.24.79.195@KRBTEST.COM (short username kudu)
I20260808 02:21:18.678411 15373 webserver.cc:460] Webserver started at http://127.24.79.195:41057/ using document root <none> and password file <none>
I20260808 02:21:18.679123 15373 fs_manager.cc:359] Metadata directory not provided
I20260808 02:21:18.679207 15373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:21:18.679445 15373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:21:18.681879 15373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "d25af570d8a54b55879a780d3d5f3505"
format_stamp: "Formatted at 2026-08-08 02:21:18 on dist-test-slave-1vmg"
server_key: "ba6463ad2a28384e0bf7c6fed0ce324e"
server_key_iv: "067dad33f8b8e8336205fd7cc11daed6"
server_key_version: "encryptionkey@0"
I20260808 02:21:18.682529 15373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "d25af570d8a54b55879a780d3d5f3505"
format_stamp: "Formatted at 2026-08-08 02:21:18 on dist-test-slave-1vmg"
server_key: "ba6463ad2a28384e0bf7c6fed0ce324e"
server_key_iv: "067dad33f8b8e8336205fd7cc11daed6"
server_key_version: "encryptionkey@0"
I20260808 02:21:18.687260 15373 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 02:21:18.690549 15393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:18.692139 15373 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:21:18.692310 15373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "d25af570d8a54b55879a780d3d5f3505"
format_stamp: "Formatted at 2026-08-08 02:21:18 on dist-test-slave-1vmg"
server_key: "ba6463ad2a28384e0bf7c6fed0ce324e"
server_key_iv: "067dad33f8b8e8336205fd7cc11daed6"
server_key_version: "encryptionkey@0"
I20260808 02:21:18.692456 15373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:21:18.720091 15373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:21:18.723970 15373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:21:18.724190 15373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:21:18.724941 15373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:21:18.726236 15373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:21:18.726317 15373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:18.726384 15373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:21:18.726413 15373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:18.737267 15373 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:38035
I20260808 02:21:18.737370 15552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:38035 every 8 connection(s)
I20260808 02:21:18.738408 15373 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:21:18.746001 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 15373
I20260808 02:21:18.746147 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:21:18.746501 24895 external_mini_cluster.cc:1428] Setting key 904e49870002126421ddecd4fae41864
Aug 08 02:21:18 dist-test-slave-1vmg krb5kdc[14862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155678, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:21:18.754455 14980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:18.740784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:54277 (local address 127.24.79.254:38659)
0808 02:21:18.740950 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:18.740954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:18.741988 (+ 1034us) server_negotiation.cc:407] Connection header received
0808 02:21:18.743474 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:18.743479 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:18.743545 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:18.743634 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:21:18.745237 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.745892 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.747482 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.747764 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.747859 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:18.750634 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:21:18.750638 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:21:18.750640 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:21:18.750676 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:21:18.752083 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:18.752691 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:18.752695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:18.752696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:18.752788 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:18.753177 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:18.753180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:18.753181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:18.753396 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:21:18.753491 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 02:21:18.754126 (+ 635us) server_negotiation.cc:299] Negotiation successful
0808 02:21:18.754279 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 02:21:18.755443 15555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:18.741180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.195:54277)
0808 02:21:18.741784 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:18.741839 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 02:21:18.743212 (+ 1373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:21:18.743852 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:21:18.743861 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:21:18.744310 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:21:18.745021 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:18.745040 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.746051 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.746054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:18.747296 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:18.747310 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.747948 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.747954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:18.748142 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:18.748800 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:21:18.748818 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:21:18.750444 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:21:18.752228 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:21:18.752237 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:21:18.752259 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:21:18.752578 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:21:18.752881 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:21:18.752885 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:21:18.752887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:21:18.753058 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:21:18.753512 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:21:18.753521 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:21:18.753834 (+ 313us) client_negotiation.cc:769] Sending connection context
0808 02:21:18.754052 (+ 218us) client_negotiation.cc:240] Negotiation successful
0808 02:21:18.754450 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":153,"threads_started":1}
I20260808 02:21:18.757015 15553 heartbeater.cc:347] Connected to a master server at 127.24.79.254:38659
I20260808 02:21:18.757326 15553 heartbeater.cc:464] Registering TS with master...
I20260808 02:21:18.757968 15553 heartbeater.cc:511] Master 127.24.79.254:38659 requested a full tablet report, sending...
I20260808 02:21:18.759394 14929 ts_manager.cc:194] Registered new tserver with Master: d25af570d8a54b55879a780d3d5f3505 (127.24.79.195:38035)
I20260808 02:21:18.760144 14929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.195@KRBTEST.COM'} at 127.24.79.195:54277
I20260808 02:21:18.761057 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:21:18.771531 14980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:18.763410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47218 (local address 127.24.79.254:38659)
0808 02:21:18.763584 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:18.763588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:18.763707 (+ 119us) server_negotiation.cc:407] Connection header received
0808 02:21:18.763843 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:18.763847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:18.763919 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:18.763993 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:21:18.764826 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.765359 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.766248 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.766491 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.766585 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:18.768104 (+ 1519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:21:18.768109 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:21:18.768111 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:21:18.768142 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:21:18.769512 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:18.770018 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:18.770023 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:18.770025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:18.770136 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:18.770429 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:18.770433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:18.770435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:18.770758 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:21:18.770900 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 02:21:18.771217 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 02:21:18.771356 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 02:21:18.775451 14929 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:21:18.778903 14929 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:21:18.795439 15569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:18.788875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:44439 (local address 127.0.0.1:53774)
0808 02:21:18.789813 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:18.789879 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 02:21:18.790024 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:21:18.791219 (+ 1195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:21:18.791227 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:21:18.791263 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:21:18.791506 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:18.791515 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.792551 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.792554 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:18.793782 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:18.793793 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.794747 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.794751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:18.794881 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:18.794963 (+ 82us) client_negotiation.cc:769] Sending connection context
0808 02:21:18.795106 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 02:21:18.795258 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"mutex_wait_us":93,"spinlock_wait_cycles":6912,"thread_start_us":134,"threads_started":1}
I20260808 02:21:18.795451 15572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:18.789603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53774 (local address 127.24.79.194:44439)
0808 02:21:18.790668 (+ 1065us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:18.790681 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:18.790706 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:21:18.790792 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:18.790797 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:18.791027 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:18.791201 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:21:18.791630 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.792404 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.793905 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.794598 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.794718 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:18.794891 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 02:21:18.795156 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 02:21:18.795287 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":945,"thread_start_us":117,"threads_started":1}
I20260808 02:21:18.796084 15575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:18.789751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:38035 (local address 127.0.0.1:59856)
0808 02:21:18.790243 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:18.790255 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:21:18.790346 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:21:18.791299 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:21:18.791301 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:21:18.791311 (+ 10us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:21:18.791518 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:18.791526 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.793196 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.793199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:18.794313 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:18.794320 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.795597 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.795602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:18.795753 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:18.795780 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 02:21:18.795852 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 02:21:18.795938 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"mutex_wait_us":130,"spinlock_wait_cycles":3328,"thread_start_us":99,"threads_started":1}
I20260808 02:21:18.796175 15578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:18.790031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59856 (local address 127.24.79.195:38035)
0808 02:21:18.790779 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:18.790791 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:18.790811 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:21:18.790876 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:18.790880 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:18.791052 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:18.791201 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:21:18.791664 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.792877 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.794445 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.795262 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.795366 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:18.795512 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 02:21:18.795924 (+ 412us) server_negotiation.cc:299] Negotiation successful
0808 02:21:18.796033 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":341,"threads_started":1}
I20260808 02:21:18.796852 15571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:18.789300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38406 (local address 127.24.79.193:41277)
0808 02:21:18.790124 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:18.790132 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:18.790443 (+ 311us) server_negotiation.cc:407] Connection header received
0808 02:21:18.790538 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:18.790543 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:18.790878 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:18.791061 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:21:18.791664 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.792877 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.794441 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.795266 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.795375 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:18.795520 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 02:21:18.796581 (+ 1061us) server_negotiation.cc:299] Negotiation successful
0808 02:21:18.796702 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":90,"threads_started":1}
I20260808 02:21:18.798480 15570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:18.789003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:41277 (local address 127.0.0.1:38406)
0808 02:21:18.790234 (+ 1231us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:18.790246 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:21:18.790345 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:21:18.791219 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:21:18.791227 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:21:18.791257 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:21:18.791518 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:18.791526 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.793021 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.793025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:18.794284 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:18.794296 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:18.795596 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:18.795600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:18.795998 (+ 398us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:18.796026 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 02:21:18.796096 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 02:21:18.798331 (+ 2235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1165,"mutex_wait_us":291,"spinlock_wait_cycles":11136,"thread_start_us":112,"threads_started":1}
I20260808 02:21:18.799227 15463 tablet_service.cc:1467] Processing CreateTablet for tablet df944712a64847c68a6885ed5e8a3fb6 (DEFAULT_TABLE table=test-table [id=1150f5b7869c4f01b474751e5dca1efb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:21:18.799217 15276 tablet_service.cc:1467] Processing CreateTablet for tablet df944712a64847c68a6885ed5e8a3fb6 (DEFAULT_TABLE table=test-table [id=1150f5b7869c4f01b474751e5dca1efb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:21:18.800308 15276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df944712a64847c68a6885ed5e8a3fb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:21:18.800319 15463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df944712a64847c68a6885ed5e8a3fb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:21:18.805439 15093 tablet_service.cc:1467] Processing CreateTablet for tablet df944712a64847c68a6885ed5e8a3fb6 (DEFAULT_TABLE table=test-table [id=1150f5b7869c4f01b474751e5dca1efb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:21:18.806834 15093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df944712a64847c68a6885ed5e8a3fb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:21:18.808434 15581 tablet_bootstrap.cc:492] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505: Bootstrap starting.
I20260808 02:21:18.808974 15583 tablet_bootstrap.cc:492] T df944712a64847c68a6885ed5e8a3fb6 P eef8385f4f864c02a4e467439d492ef1: Bootstrap starting.
I20260808 02:21:18.811517 15581 tablet_bootstrap.cc:654] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505: Neither blocks nor log segments found. Creating new log.
I20260808 02:21:18.811532 15583 tablet_bootstrap.cc:654] T df944712a64847c68a6885ed5e8a3fb6 P eef8385f4f864c02a4e467439d492ef1: Neither blocks nor log segments found. Creating new log.
I20260808 02:21:18.812529 15581 log.cc:824] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505: Log is configured to *not* fsync() on all Append() calls
I20260808 02:21:18.812529 15583 log.cc:824] T df944712a64847c68a6885ed5e8a3fb6 P eef8385f4f864c02a4e467439d492ef1: Log is configured to *not* fsync() on all Append() calls
I20260808 02:21:18.814990 15583 tablet_bootstrap.cc:492] T df944712a64847c68a6885ed5e8a3fb6 P eef8385f4f864c02a4e467439d492ef1: No bootstrap required, opened a new log
I20260808 02:21:18.814990 15581 tablet_bootstrap.cc:492] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505: No bootstrap required, opened a new log
I20260808 02:21:18.814990 15586 tablet_bootstrap.cc:492] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2: Bootstrap starting.
I20260808 02:21:18.815233 15583 ts_tablet_manager.cc:1397] T df944712a64847c68a6885ed5e8a3fb6 P eef8385f4f864c02a4e467439d492ef1: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260808 02:21:18.815233 15581 ts_tablet_manager.cc:1397] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260808 02:21:18.817234 15586 tablet_bootstrap.cc:654] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2: Neither blocks nor log segments found. Creating new log.
I20260808 02:21:18.818109 15586 log.cc:824] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2: Log is configured to *not* fsync() on all Append() calls
I20260808 02:21:18.819484 15583 raft_consensus.cc:348] T df944712a64847c68a6885ed5e8a3fb6 P eef8385f4f864c02a4e467439d492ef1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } } peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } }
I20260808 02:21:18.819484 15581 raft_consensus.cc:348] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } } peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } }
I20260808 02:21:18.819876 15581 raft_consensus.cc:374] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:21:18.819876 15583 raft_consensus.cc:374] T df944712a64847c68a6885ed5e8a3fb6 P eef8385f4f864c02a4e467439d492ef1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:21:18.819973 15583 raft_consensus.cc:729] T df944712a64847c68a6885ed5e8a3fb6 P eef8385f4f864c02a4e467439d492ef1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eef8385f4f864c02a4e467439d492ef1, State: Initialized, Role: FOLLOWER
I20260808 02:21:18.819986 15581 raft_consensus.cc:729] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d25af570d8a54b55879a780d3d5f3505, State: Initialized, Role: FOLLOWER
I20260808 02:21:18.820446 15586 tablet_bootstrap.cc:492] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2: No bootstrap required, opened a new log
I20260808 02:21:18.820600 15581 consensus_queue.cc:260] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } } peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } }
I20260808 02:21:18.820609 15583 consensus_queue.cc:260] T df944712a64847c68a6885ed5e8a3fb6 P eef8385f4f864c02a4e467439d492ef1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } } peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } }
I20260808 02:21:18.820736 15586 ts_tablet_manager.cc:1397] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 02:21:18.821806 15581 ts_tablet_manager.cc:1428] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 02:21:18.821806 15583 ts_tablet_manager.cc:1428] T df944712a64847c68a6885ed5e8a3fb6 P eef8385f4f864c02a4e467439d492ef1: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 02:21:18.821986 15366 heartbeater.cc:503] Master 127.24.79.254:38659 was elected leader, sending a full tablet report...
I20260808 02:21:18.822317 15553 heartbeater.cc:503] Master 127.24.79.254:38659 was elected leader, sending a full tablet report...
I20260808 02:21:18.825608 15586 raft_consensus.cc:348] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } } peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } }
I20260808 02:21:18.825961 15586 raft_consensus.cc:374] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:21:18.826079 15586 raft_consensus.cc:729] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df902d8757214a8da739a4da5255e2e2, State: Initialized, Role: FOLLOWER
I20260808 02:21:18.826633 15586 consensus_queue.cc:260] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } } peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } }
I20260808 02:21:18.827628 15176 heartbeater.cc:503] Master 127.24.79.254:38659 was elected leader, sending a full tablet report...
I20260808 02:21:18.827891 15586 ts_tablet_manager.cc:1428] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2: Time spent starting tablet: real 0.007s user 0.000s sys 0.006s
W20260808 02:21:18.972811 15367 tablet.cc:2369] T df944712a64847c68a6885ed5e8a3fb6 P eef8385f4f864c02a4e467439d492ef1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:21:18.977744 15177 tablet.cc:2369] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:21:18.989657 15554 tablet.cc:2369] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:21:19.026516 15588 raft_consensus.cc:482] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:21:19.026831 15588 raft_consensus.cc:504] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } } peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } }
I20260808 02:21:19.028374 15588 leader_election.cc:288] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eef8385f4f864c02a4e467439d492ef1 (127.24.79.194:44439), df902d8757214a8da739a4da5255e2e2 (127.24.79.193:41277)
I20260808 02:21:19.032430 15555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:19.028710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:44439 (local address 127.24.79.195:44465)
0808 02:21:19.028834 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:19.028849 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:21:19.029003 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:21:19.029282 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:21:19.029286 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:21:19.029310 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:21:19.029444 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:19.029451 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:19.030387 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:19.030390 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:19.031290 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:19.031301 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:19.031968 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:19.031971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:19.032086 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:19.032110 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 02:21:19.032181 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 02:21:19.032269 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 02:21:19.032430 15572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:19.028797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:44465 (local address 127.24.79.194:44439)
0808 02:21:19.028930 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:19.028934 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:19.028951 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:21:19.029112 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:19.029114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:19.029168 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:19.029243 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:21:19.029573 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:19.030285 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:19.031431 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:19.031847 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:19.031968 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:19.032034 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 02:21:19.032213 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 02:21:19.032297 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:21:19.033494 15592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:19.028710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:41277 (local address 127.24.79.195:60925)
0808 02:21:19.029490 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:19.029504 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:21:19.029622 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:21:19.029893 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:21:19.029895 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:21:19.029914 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:21:19.030040 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:19.030046 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:19.030892 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:19.030896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:19.031973 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:19.031986 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:19.032897 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:19.032900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:19.033085 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:19.033179 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 02:21:19.033276 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 02:21:19.033357 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"mutex_wait_us":362,"thread_start_us":145,"threads_started":1}
I20260808 02:21:19.033607 15571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:19.028811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:60925 (local address 127.24.79.193:41277)
0808 02:21:19.028930 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:19.028934 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:19.029637 (+ 703us) server_negotiation.cc:407] Connection header received
0808 02:21:19.029749 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:19.029751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:19.029798 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:19.029867 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:21:19.030227 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:19.030745 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:19.032155 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:19.032720 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:19.032852 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:19.032907 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 02:21:19.033361 (+ 454us) server_negotiation.cc:299] Negotiation successful
0808 02:21:19.033473 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 02:21:19.033738 15302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df944712a64847c68a6885ed5e8a3fb6" candidate_uuid: "d25af570d8a54b55879a780d3d5f3505" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eef8385f4f864c02a4e467439d492ef1" is_pre_election: true
I20260808 02:21:19.034147 15302 raft_consensus.cc:2454] T df944712a64847c68a6885ed5e8a3fb6 P eef8385f4f864c02a4e467439d492ef1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d25af570d8a54b55879a780d3d5f3505 in term 0.
I20260808 02:21:19.034798 15402 leader_election.cc:302] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d25af570d8a54b55879a780d3d5f3505, eef8385f4f864c02a4e467439d492ef1; no voters:
I20260808 02:21:19.035193 15588 raft_consensus.cc:2781] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:21:19.035300 15588 raft_consensus.cc:482] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:21:19.035264 15119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df944712a64847c68a6885ed5e8a3fb6" candidate_uuid: "d25af570d8a54b55879a780d3d5f3505" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df902d8757214a8da739a4da5255e2e2" is_pre_election: true
I20260808 02:21:19.035357 15588 raft_consensus.cc:3037] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:21:19.035588 15119 raft_consensus.cc:2454] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d25af570d8a54b55879a780d3d5f3505 in term 0.
I20260808 02:21:19.036881 15588 raft_consensus.cc:504] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } } peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } }
I20260808 02:21:19.037388 15588 leader_election.cc:288] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505 [CANDIDATE]: Term 1 election: Requested vote from peers eef8385f4f864c02a4e467439d492ef1 (127.24.79.194:44439), df902d8757214a8da739a4da5255e2e2 (127.24.79.193:41277)
I20260808 02:21:19.037810 15119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df944712a64847c68a6885ed5e8a3fb6" candidate_uuid: "d25af570d8a54b55879a780d3d5f3505" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df902d8757214a8da739a4da5255e2e2"
I20260808 02:21:19.037899 15302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df944712a64847c68a6885ed5e8a3fb6" candidate_uuid: "d25af570d8a54b55879a780d3d5f3505" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eef8385f4f864c02a4e467439d492ef1"
I20260808 02:21:19.037953 15119 raft_consensus.cc:3037] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:21:19.038053 15302 raft_consensus.cc:3037] T df944712a64847c68a6885ed5e8a3fb6 P eef8385f4f864c02a4e467439d492ef1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:21:19.039304 15119 raft_consensus.cc:2454] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d25af570d8a54b55879a780d3d5f3505 in term 1.
I20260808 02:21:19.039445 15302 raft_consensus.cc:2454] T df944712a64847c68a6885ed5e8a3fb6 P eef8385f4f864c02a4e467439d492ef1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d25af570d8a54b55879a780d3d5f3505 in term 1.
I20260808 02:21:19.039743 15394 leader_election.cc:302] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d25af570d8a54b55879a780d3d5f3505, df902d8757214a8da739a4da5255e2e2; no voters:
I20260808 02:21:19.040045 15588 raft_consensus.cc:2781] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:21:19.040323 15588 raft_consensus.cc:686] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505 [term 1 LEADER]: Becoming Leader. State: Replica: d25af570d8a54b55879a780d3d5f3505, State: Running, Role: LEADER
I20260808 02:21:19.040782 15588 consensus_queue.cc:237] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } } peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } }
I20260808 02:21:19.044536 14928 catalog_manager.cc:5495] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505 reported cstate change: term changed from 0 to 1, leader changed from <none> to d25af570d8a54b55879a780d3d5f3505 (127.24.79.195). New cstate: current_term: 1 leader_uuid: "d25af570d8a54b55879a780d3d5f3505" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } health_report { overall_health: HEALTHY } } }
I20260808 02:21:19.095405 15578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:19.091156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59858 (local address 127.24.79.195:38035)
0808 02:21:19.091384 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:19.091388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:19.091414 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:21:19.091509 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:19.091512 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:19.091575 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:19.091683 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:21:19.091985 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:19.092528 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:19.093450 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:19.093708 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:19.093805 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:19.094260 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:21:19.094787 (+ 527us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:21:19.094945 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 02:21:19.095147 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 02:21:19.095240 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 02:21:19.103050 15302 raft_consensus.cc:1261] T df944712a64847c68a6885ed5e8a3fb6 P eef8385f4f864c02a4e467439d492ef1 [term 1 FOLLOWER]: Refusing update from remote peer d25af570d8a54b55879a780d3d5f3505: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:21:19.103050 15119 raft_consensus.cc:1261] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Refusing update from remote peer d25af570d8a54b55879a780d3d5f3505: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:21:19.104087 15593 consensus_queue.cc:1035] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:21:19.104365 15588 consensus_queue.cc:1035] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:21:19.114589 15600 mvcc.cc:205] Tried to move back new op lower bound from 7316093661595160576 to 7316093661354106880. Current Snapshot: MvccSnapshot[applied={T|T < 7316093661595160576}]
I20260808 02:21:19.115612 15599 mvcc.cc:205] Tried to move back new op lower bound from 7316093661595160576 to 7316093661354106880. Current Snapshot: MvccSnapshot[applied={T|T < 7316093661595160576}]
I20260808 02:21:19.126157 14928 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:21:19.126374 14928 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:21:19.129433 14928 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0: Sending DeleteTablet for 3 replicas of tablet df944712a64847c68a6885ed5e8a3fb6
I20260808 02:21:19.130534 15463 tablet_service.cc:1514] Processing DeleteTablet for tablet df944712a64847c68a6885ed5e8a3fb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:21:19 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:59856
I20260808 02:21:19.130528 15093 tablet_service.cc:1514] Processing DeleteTablet for tablet df944712a64847c68a6885ed5e8a3fb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:21:19 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:38406
I20260808 02:21:19.130736 15276 tablet_service.cc:1514] Processing DeleteTablet for tablet df944712a64847c68a6885ed5e8a3fb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:21:19 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:53774
I20260808 02:21:19.131239 15611 tablet_replica.cc:330] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505: stopping tablet replica
I20260808 02:21:19.131239 15612 tablet_replica.cc:330] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2: stopping tablet replica
I20260808 02:21:19.131302 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 15010
I20260808 02:21:19.131654 15612 raft_consensus.cc:2229] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:21:19.131654 15611 raft_consensus.cc:2229] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:21:19.134281 15613 tablet_replica.cc:330] T df944712a64847c68a6885ed5e8a3fb6 P eef8385f4f864c02a4e467439d492ef1: stopping tablet replica
I20260808 02:21:19.134483 15611 raft_consensus.cc:2258] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:21:19.134732 15613 raft_consensus.cc:2229] T df944712a64847c68a6885ed5e8a3fb6 P eef8385f4f864c02a4e467439d492ef1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:21:19.135124 15613 raft_consensus.cc:2258] T df944712a64847c68a6885ed5e8a3fb6 P eef8385f4f864c02a4e467439d492ef1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:21:19.143306 15613 ts_tablet_manager.cc:1905] T df944712a64847c68a6885ed5e8a3fb6 P eef8385f4f864c02a4e467439d492ef1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:21:19.143306 15611 ts_tablet_manager.cc:1905] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:21:19.146665 15611 ts_tablet_manager.cc:1918] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:21:19.146737 15613 ts_tablet_manager.cc:1918] T df944712a64847c68a6885ed5e8a3fb6 P eef8385f4f864c02a4e467439d492ef1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:21:19.146816 15611 log.cc:1196] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/df944712a64847c68a6885ed5e8a3fb6
I20260808 02:21:19.146837 15613 log.cc:1196] T df944712a64847c68a6885ed5e8a3fb6 P eef8385f4f864c02a4e467439d492ef1: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/df944712a64847c68a6885ed5e8a3fb6
I20260808 02:21:19.147222 15613 ts_tablet_manager.cc:1939] T df944712a64847c68a6885ed5e8a3fb6 P eef8385f4f864c02a4e467439d492ef1: Deleting consensus metadata
I20260808 02:21:19.147274 15611 ts_tablet_manager.cc:1939] T df944712a64847c68a6885ed5e8a3fb6 P d25af570d8a54b55879a780d3d5f3505: Deleting consensus metadata
I20260808 02:21:19.148846 14908 catalog_manager.cc:4841] TS d25af570d8a54b55879a780d3d5f3505 (127.24.79.195:38035): tablet df944712a64847c68a6885ed5e8a3fb6 (table test-table [id=1150f5b7869c4f01b474751e5dca1efb]) successfully deleted
I20260808 02:21:19.149086 14908 catalog_manager.cc:4841] TS eef8385f4f864c02a4e467439d492ef1 (127.24.79.194:44439): tablet df944712a64847c68a6885ed5e8a3fb6 (table test-table [id=1150f5b7869c4f01b474751e5dca1efb]) successfully deleted
W20260808 02:21:19.160656 14905 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.79.193:41277 (error 108)
I20260808 02:21:19.161366 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 15183
I20260808 02:21:19.171999 15570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:19.161608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:41277 (local address 127.0.0.1:38418)
0808 02:21:19.171715 (+ 10107us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:19.171871 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.193:41277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9959}
W20260808 02:21:19.174360 14905 catalog_manager.cc:4584] TS df902d8757214a8da739a4da5255e2e2 (127.24.79.193:41277): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet df944712a64847c68a6885ed5e8a3fb6: Network error: Client connection negotiation failed: client connection to 127.24.79.193:41277: connect: Connection refused (error 111)
I20260808 02:21:19.193419 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 15373
I20260808 02:21:19.208765 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 14879
I20260808 02:21:39.224774 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:38659
--webserver_interface=127.24.79.254
--webserver_port=42817
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:40593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:38659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:21:39.358887 15615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:21:39.359460 15615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:21:39.359566 15615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:21:39.364207 15615 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 02:21:39.364290 15615 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 02:21:39.364313 15615 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 02:21:39.364332 15615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:21:39.364357 15615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:21:39.364393 15615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:21:39.364425 15615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:21:39.364457 15615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:21:39.370478 15615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:38659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:38659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=42817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:21:39.371865 15615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:21:39.373282 15615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:21:39.390300 15625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:21:39.390352 15615 server_base.cc:1034] running on GCE node
W20260808 02:21:39.390288 15626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:21:39.390288 15629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:21:39.391456 15615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:21:39.392944 15615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:21:39.394191 15615 hybrid_clock.cc:648] HybridClock initialized: now 1786155699394157 us; error 53 us; skew 500 ppm
Aug 08 02:21:39 dist-test-slave-1vmg krb5kdc[14862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155699, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:21:39.401854 15615 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:21:39.403344 15615 webserver.cc:460] Webserver started at http://127.24.79.254:42817/ using document root <none> and password file <none>
I20260808 02:21:39.404080 15615 fs_manager.cc:359] Metadata directory not provided
I20260808 02:21:39.404207 15615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:21:39.409123 15615 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 02:21:39.412014 15641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:39.413820 15615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:21:39.414017 15615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "69b642c340c5439dafce5fb8caefdce0"
format_stamp: "Formatted at 2026-08-08 02:21:17 on dist-test-slave-1vmg"
server_key: "67108216d134b7ee5cfcd1d43a781919"
server_key_iv: "b0ea4e4b14fb21ecbbe9305acb627156"
server_key_version: "encryptionkey@0"
I20260808 02:21:39.414871 15615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:21:39.430740 15615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:21:39.435822 15615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:21:39.436072 15615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:21:39.445082 15615 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:38659
I20260808 02:21:39.445403 15720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:38659 every 8 connection(s)
I20260808 02:21:39.446646 15615 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:21:39.452558 15721 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:21:39.454813 15721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0: Bootstrap starting.
I20260808 02:21:39.455226 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 15615
I20260808 02:21:39.455739 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:41277
--local_ip_for_outbound_sockets=127.24.79.193
--tserver_master_addrs=127.24.79.254:38659
--webserver_port=36793
--webserver_interface=127.24.79.193
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:40593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:21:39.475829 15721 log.cc:824] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0: Log is configured to *not* fsync() on all Append() calls
I20260808 02:21:39.486550 15721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:21:39.487056 15721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0: Bootstrap complete.
I20260808 02:21:39.491642 15721 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b642c340c5439dafce5fb8caefdce0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38659 } }
I20260808 02:21:39.492154 15721 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69b642c340c5439dafce5fb8caefdce0, State: Initialized, Role: FOLLOWER
I20260808 02:21:39.492729 15721 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b642c340c5439dafce5fb8caefdce0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38659 } }
I20260808 02:21:39.492873 15721 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:21:39.492960 15721 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:21:39.493067 15721 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:21:39.495509 15721 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b642c340c5439dafce5fb8caefdce0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38659 } }
I20260808 02:21:39.495922 15721 leader_election.cc:302] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69b642c340c5439dafce5fb8caefdce0; no voters:
I20260808 02:21:39.496299 15721 leader_election.cc:288] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 02:21:39.496434 15725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 02:21:39.496652 15725 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 2 LEADER]: Becoming Leader. State: Replica: 69b642c340c5439dafce5fb8caefdce0, State: Running, Role: LEADER
I20260808 02:21:39.497088 15725 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b642c340c5439dafce5fb8caefdce0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38659 } }
I20260808 02:21:39.497843 15721 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:21:39.498812 15726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69b642c340c5439dafce5fb8caefdce0. Latest consensus state: current_term: 2 leader_uuid: "69b642c340c5439dafce5fb8caefdce0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b642c340c5439dafce5fb8caefdce0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38659 } } }
I20260808 02:21:39.499008 15726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [sys.catalog]: This master's current role is: LEADER
I20260808 02:21:39.499429 15738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:21:39.499432 15727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "69b642c340c5439dafce5fb8caefdce0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b642c340c5439dafce5fb8caefdce0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38659 } } }
I20260808 02:21:39.499614 15727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [sys.catalog]: This master's current role is: LEADER
I20260808 02:21:39.505162 15738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:21:39.505990 15738 catalog_manager.cc:1259] Loaded cluster ID: 071d8474e0dd413bb7ec01c0edfdb77b
I20260808 02:21:39.506067 15738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:21:39.508206 15738 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:21:39.508970 15738 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0: Loaded TSK: 0
I20260808 02:21:39.523176 15738 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0: Generated new TSK 1
I20260808 02:21:39.523540 15738 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 02:21:39.620491 15723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:21:39.620944 15723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:21:39.621071 15723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:21:39.628062 15723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:21:39.628165 15723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:21:39.628255 15723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:21:39.637118 15723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:41277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=36793
--tserver_master_addrs=127.24.79.254:38659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:21:39.638705 15723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:21:39.640271 15723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:21:39.652401 15761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:21:39.652401 15760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:21:39.653115 15763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:21:39.653093 15723 server_base.cc:1034] running on GCE node
I20260808 02:21:39.654026 15723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:21:39.654877 15723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:21:39.656103 15723 hybrid_clock.cc:648] HybridClock initialized: now 1786155699656065 us; error 64 us; skew 500 ppm
Aug 08 02:21:39 dist-test-slave-1vmg krb5kdc[14862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155699, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:21:39.659915 15723 init.cc:377] Logged in from keytab as kudu/127.24.79.193@KRBTEST.COM (short username kudu)
I20260808 02:21:39.661442 15723 webserver.cc:460] Webserver started at http://127.24.79.193:36793/ using document root <none> and password file <none>
I20260808 02:21:39.662319 15723 fs_manager.cc:359] Metadata directory not provided
I20260808 02:21:39.662463 15723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:21:39.668640 15723 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:21:39.672042 15773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:39.673836 15723 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:21:39.674003 15723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "df902d8757214a8da739a4da5255e2e2"
format_stamp: "Formatted at 2026-08-08 02:21:18 on dist-test-slave-1vmg"
server_key: "ddf9554af2f50e833f96134e1208266e"
server_key_iv: "8f1234e252ef67825aacaa24fcb4551f"
server_key_version: "encryptionkey@0"
I20260808 02:21:39.674786 15723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:21:39.714087 15723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:21:39.718104 15723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:21:39.718523 15723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:21:39.719414 15723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:21:39.720954 15783 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 02:21:39.726205 15723 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 02:21:39.726339 15723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260808 02:21:39.726408 15723 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 02:21:39.729008 15723 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 02:21:39.729095 15723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.000s
I20260808 02:21:39.729576 15783 tablet_bootstrap.cc:492] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2: Bootstrap starting.
I20260808 02:21:39.740756 15723 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:41277
I20260808 02:21:39.740840 15948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:41277 every 8 connection(s)
I20260808 02:21:39.741963 15723 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:21:39.747903 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 15723
I20260808 02:21:39.748620 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:44439
--local_ip_for_outbound_sockets=127.24.79.194
--tserver_master_addrs=127.24.79.254:38659
--webserver_port=38163
--webserver_interface=127.24.79.194
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:40593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:21:39.751284 15783 log.cc:824] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:21:39 dist-test-slave-1vmg krb5kdc[14862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155699, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:21:39.761363 15783 tablet_bootstrap.cc:492] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:21:39.761860 15783 tablet_bootstrap.cc:492] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2: Bootstrap complete.
I20260808 02:21:39.763356 15783 ts_tablet_manager.cc:1397] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2: Time spent bootstrapping tablet: real 0.034s user 0.025s sys 0.008s
I20260808 02:21:39.765952 15954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:39.744951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:46227 (local address 127.24.79.254:38659)
0808 02:21:39.745500 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:39.745601 (+ 101us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:39.745887 (+ 286us) server_negotiation.cc:407] Connection header received
0808 02:21:39.747394 (+ 1507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:39.747412 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:39.747978 (+ 566us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:39.748482 (+ 504us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:21:39.750834 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:39.752246 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:39.753989 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:39.754800 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:39.755048 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:39.758642 (+ 3594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:21:39.758651 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:21:39.758671 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:21:39.758717 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:21:39.761148 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:39.762141 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:39.762154 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:39.762161 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:39.762385 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:39.763008 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:39.763012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:39.763014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:39.763600 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:21:39.763749 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 02:21:39.764550 (+ 801us) server_negotiation.cc:299] Negotiation successful
0808 02:21:39.764919 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":171,"threads_started":1}
I20260808 02:21:39.765995 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:39.744965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.193:46227)
0808 02:21:39.745658 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:39.745725 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 02:21:39.746650 (+ 925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:21:39.748631 (+ 1981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:21:39.748642 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:21:39.749548 (+ 906us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:21:39.750499 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:39.750526 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:39.752445 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:39.752448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:39.753800 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:39.753815 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:39.755028 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:39.755033 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:39.755250 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:39.756004 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:21:39.756027 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:21:39.758300 (+ 2273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:21:39.761349 (+ 3049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:21:39.761358 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:21:39.761376 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:21:39.761898 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:21:39.762571 (+ 673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:21:39.762624 (+ 53us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:21:39.762626 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:21:39.762862 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:21:39.763765 (+ 903us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:21:39.763776 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:21:39.764134 (+ 358us) client_negotiation.cc:769] Sending connection context
0808 02:21:39.764385 (+ 251us) client_negotiation.cc:240] Negotiation successful
0808 02:21:39.764914 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":219,"threads_started":1}
I20260808 02:21:39.767099 15783 raft_consensus.cc:348] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } } peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } }
I20260808 02:21:39.767805 15783 raft_consensus.cc:729] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: df902d8757214a8da739a4da5255e2e2, State: Initialized, Role: FOLLOWER
I20260808 02:21:39.768407 15783 consensus_queue.cc:260] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } } peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } }
I20260808 02:21:39.769625 15783 ts_tablet_manager.cc:1428] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 02:21:39.769701 15950 heartbeater.cc:347] Connected to a master server at 127.24.79.254:38659
I20260808 02:21:39.769948 15950 heartbeater.cc:464] Registering TS with master...
I20260808 02:21:39.770507 15950 heartbeater.cc:511] Master 127.24.79.254:38659 requested a full tablet report, sending...
I20260808 02:21:39.773926 15663 ts_manager.cc:194] Registered new tserver with Master: df902d8757214a8da739a4da5255e2e2 (127.24.79.193:41277)
I20260808 02:21:39.778882 15663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.193@KRBTEST.COM'} at 127.24.79.193:46227
I20260808 02:21:39.782424 15950 heartbeater.cc:503] Master 127.24.79.254:38659 was elected leader, sending a full tablet report...
Aug 08 02:21:39 dist-test-slave-1vmg krb5kdc[14862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155699, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for kudu/127.24.79.193@KRBTEST.COM
I20260808 02:21:39.790210 15962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:39.778226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35238 (local address 127.24.79.193:41277)
0808 02:21:39.778761 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:39.778774 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:39.779437 (+ 663us) server_negotiation.cc:407] Connection header received
0808 02:21:39.779518 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:39.779523 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:39.779717 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:39.779867 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:21:39.780729 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:39.781700 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:39.782648 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:39.783147 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:39.783262 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:39.785839 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:21:39.785851 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:21:39.785865 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:21:39.785894 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:21:39.787919 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:39.788566 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:39.788572 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:39.788573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:39.788678 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:39.788992 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:39.788996 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:39.788997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:39.789305 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:21:39.789447 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 02:21:39.789856 (+ 409us) server_negotiation.cc:299] Negotiation successful
0808 02:21:39.790050 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":409,"thread_start_us":132,"threads_started":1}
I20260808 02:21:39.790210 15961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:39.778061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:41277 (local address 127.0.0.1:35238)
0808 02:21:39.779068 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:39.779123 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 02:21:39.779304 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:21:39.779905 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:21:39.779911 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:21:39.780460 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:21:39.780593 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:39.780602 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:39.781821 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:39.781825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:39.782493 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:39.782505 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:39.783287 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:39.783291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:39.783428 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:39.784028 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:21:39.784042 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:21:39.785657 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:21:39.788086 (+ 2429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:21:39.788104 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:21:39.788111 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:21:39.788421 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:21:39.788788 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:21:39.788792 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:21:39.788794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:21:39.788893 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:21:39.789443 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:21:39.789450 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:21:39.789597 (+ 147us) client_negotiation.cc:769] Sending connection context
0808 02:21:39.789732 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 02:21:39.790004 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":799,"thread_start_us":224,"threads_started":1}
I20260808 02:21:39.791764 15849 tablet_service.cc:1514] Processing DeleteTablet for tablet df944712a64847c68a6885ed5e8a3fb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:21:19 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:35238
I20260808 02:21:39.793179 15963 tablet_replica.cc:330] stopping tablet replica
I20260808 02:21:39.793570 15963 raft_consensus.cc:2229] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:21:39.793859 15963 raft_consensus.cc:2258] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:21:39.795423 15963 ts_tablet_manager.cc:1905] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:21:39.800001 15963 ts_tablet_manager.cc:1918] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:21:39.800136 15963 log.cc:1196] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/wal/wals/df944712a64847c68a6885ed5e8a3fb6
I20260808 02:21:39.800401 15963 ts_tablet_manager.cc:1939] T df944712a64847c68a6885ed5e8a3fb6 P df902d8757214a8da739a4da5255e2e2: Deleting consensus metadata
I20260808 02:21:39.802696 15642 catalog_manager.cc:4841] TS df902d8757214a8da739a4da5255e2e2 (127.24.79.193:41277): tablet df944712a64847c68a6885ed5e8a3fb6 (table test-table [id=1150f5b7869c4f01b474751e5dca1efb]) successfully deleted
W20260808 02:21:39.896247 15956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:21:39.896595 15956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:21:39.896644 15956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:21:39.901319 15956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:21:39.901417 15956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:21:39.901481 15956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:21:39.907887 15956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:44439
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=38163
--tserver_master_addrs=127.24.79.254:38659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:21:39.909134 15956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:21:39.910267 15956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:21:39.921656 15973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:21:39.921798 15956 server_base.cc:1034] running on GCE node
W20260808 02:21:39.921640 15972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:21:39.921610 15975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:21:39.922456 15956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:21:39.923269 15956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:21:39.924474 15956 hybrid_clock.cc:648] HybridClock initialized: now 1786155699924441 us; error 45 us; skew 500 ppm
Aug 08 02:21:39 dist-test-slave-1vmg krb5kdc[14862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155699, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:21:39.927982 15956 init.cc:377] Logged in from keytab as kudu/127.24.79.194@KRBTEST.COM (short username kudu)
I20260808 02:21:39.929088 15956 webserver.cc:460] Webserver started at http://127.24.79.194:38163/ using document root <none> and password file <none>
I20260808 02:21:39.929788 15956 fs_manager.cc:359] Metadata directory not provided
I20260808 02:21:39.929888 15956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:21:39.934851 15956 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:21:39.937507 15989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:39.939292 15956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:21:39.939438 15956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "eef8385f4f864c02a4e467439d492ef1"
format_stamp: "Formatted at 2026-08-08 02:21:18 on dist-test-slave-1vmg"
server_key: "0983995ffa15abd31666070d87bbab2e"
server_key_iv: "c2119a443dc760383193fba777d0f6d4"
server_key_version: "encryptionkey@0"
I20260808 02:21:39.940073 15956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:21:39.989856 15956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:21:39.994508 15956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:21:39.994730 15956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:21:39.995561 15956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:21:39.996624 15956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:21:39.996686 15956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:39.996735 15956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:21:39.996753 15956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:40.008183 15956 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:44439
I20260808 02:21:40.008303 16140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:44439 every 8 connection(s)
I20260808 02:21:40.009562 15956 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:21:40.009977 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 15956
I20260808 02:21:40.010620 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:38035
--local_ip_for_outbound_sockets=127.24.79.195
--tserver_master_addrs=127.24.79.254:38659
--webserver_port=41057
--webserver_interface=127.24.79.195
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:40593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:21:40 dist-test-slave-1vmg krb5kdc[14862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155699, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:21:40.028664 15954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:40.012682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:58315 (local address 127.24.79.254:38659)
0808 02:21:40.012876 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:40.012883 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:40.014094 (+ 1211us) server_negotiation.cc:407] Connection header received
0808 02:21:40.016296 (+ 2202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:40.016301 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:40.016394 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:40.016531 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:21:40.018101 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.019018 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.020669 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.021066 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.021220 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:40.024286 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:21:40.024290 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:21:40.024298 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:21:40.024330 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:21:40.026120 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:40.026801 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:40.026805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:40.026806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:40.026904 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:40.027334 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:40.027338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:40.027340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:40.027559 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:21:40.027660 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 02:21:40.028298 (+ 638us) server_negotiation.cc:299] Negotiation successful
0808 02:21:40.028471 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 02:21:40.029647 16144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:40.013122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.194:58315)
0808 02:21:40.013830 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:40.013910 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0808 02:21:40.015893 (+ 1983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:21:40.016816 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:21:40.016825 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:21:40.017276 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:21:40.017869 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:40.017888 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.019295 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.019300 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:40.020506 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:40.020522 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.021384 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.021389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:40.021590 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:40.022285 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:21:40.022307 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:21:40.024040 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:21:40.026291 (+ 2251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:21:40.026300 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:21:40.026327 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:21:40.026664 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:21:40.027022 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:21:40.027025 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:21:40.027028 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:21:40.027225 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:21:40.027685 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:21:40.027694 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:21:40.028010 (+ 316us) client_negotiation.cc:769] Sending connection context
0808 02:21:40.028210 (+ 200us) client_negotiation.cc:240] Negotiation successful
0808 02:21:40.028629 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":151,"threads_started":1}
I20260808 02:21:40.031271 16141 heartbeater.cc:347] Connected to a master server at 127.24.79.254:38659
I20260808 02:21:40.031582 16141 heartbeater.cc:464] Registering TS with master...
I20260808 02:21:40.032198 16141 heartbeater.cc:511] Master 127.24.79.254:38659 requested a full tablet report, sending...
I20260808 02:21:40.033532 15663 ts_manager.cc:194] Registered new tserver with Master: eef8385f4f864c02a4e467439d492ef1 (127.24.79.194:44439)
I20260808 02:21:40.034390 15663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.194@KRBTEST.COM'} at 127.24.79.194:58315
W20260808 02:21:40.146981 16143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:21:40.147323 16143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:21:40.147370 16143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:21:40.151906 16143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:21:40.152004 16143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:21:40.152074 16143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:21:40.157996 16143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:38035
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=41057
--tserver_master_addrs=127.24.79.254:38659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:21:40.159296 16143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:21:40.160507 16143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:21:40.171656 16152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:21:40.171689 16150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:21:40.171861 16143 server_base.cc:1034] running on GCE node
W20260808 02:21:40.171691 16155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:21:40.172564 16143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:21:40.173444 16143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:21:40.174669 16143 hybrid_clock.cc:648] HybridClock initialized: now 1786155700174645 us; error 42 us; skew 500 ppm
Aug 08 02:21:40 dist-test-slave-1vmg krb5kdc[14862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155700, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:21:40.178310 16143 init.cc:377] Logged in from keytab as kudu/127.24.79.195@KRBTEST.COM (short username kudu)
I20260808 02:21:40.179574 16143 webserver.cc:460] Webserver started at http://127.24.79.195:41057/ using document root <none> and password file <none>
I20260808 02:21:40.180274 16143 fs_manager.cc:359] Metadata directory not provided
I20260808 02:21:40.180379 16143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:21:40.185252 16143 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:21:40.188095 16165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:40.189914 16143 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:21:40.190075 16143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "d25af570d8a54b55879a780d3d5f3505"
format_stamp: "Formatted at 2026-08-08 02:21:18 on dist-test-slave-1vmg"
server_key: "ba6463ad2a28384e0bf7c6fed0ce324e"
server_key_iv: "067dad33f8b8e8336205fd7cc11daed6"
server_key_version: "encryptionkey@0"
I20260808 02:21:40.190711 16143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:21:40.202050 16143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:21:40.206075 16143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:21:40.206279 16143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:21:40.207023 16143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:21:40.207991 16143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:21:40.208045 16143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:40.208093 16143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:21:40.208108 16143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:21:40.218700 16143 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:38035
I20260808 02:21:40.218743 16320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:38035 every 8 connection(s)
I20260808 02:21:40.220125 16143 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:21:40.221556 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 16143
I20260808 02:21:40.232879 15954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:40.222451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60656 (local address 127.24.79.254:38659)
0808 02:21:40.222613 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:40.222625 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:40.222986 (+ 361us) server_negotiation.cc:407] Connection header received
0808 02:21:40.223132 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:40.223135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:40.223193 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:40.223299 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:21:40.224399 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.224930 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.226485 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.226811 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.226951 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:40.228522 (+ 1571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:21:40.228527 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:21:40.228529 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:21:40.228568 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:21:40.230433 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:40.231046 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:40.231049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:40.231051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:40.231159 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:40.231506 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:40.231510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:40.231511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:40.231752 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:21:40.231992 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 02:21:40.232512 (+ 520us) server_negotiation.cc:299] Negotiation successful
0808 02:21:40.232677 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 02:21:40 dist-test-slave-1vmg krb5kdc[14862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155700, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:21:40.239926 16326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:40.222536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:50189 (local address 127.24.79.254:38659)
0808 02:21:40.222799 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:40.222803 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:40.223954 (+ 1151us) server_negotiation.cc:407] Connection header received
0808 02:21:40.226563 (+ 2609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:40.226566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:40.226643 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:40.226746 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:21:40.228550 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.229287 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.230751 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.231134 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.231225 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:40.234840 (+ 3615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:21:40.234845 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:21:40.234848 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:21:40.234878 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:21:40.236692 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:40.237515 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:40.237519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:40.237520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:40.237636 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:40.238386 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:40.238389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:40.238391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:40.238644 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:21:40.238747 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 02:21:40.239577 (+ 830us) server_negotiation.cc:299] Negotiation successful
0808 02:21:40.239775 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":78,"threads_started":1}
I20260808 02:21:40.240926 16329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:40.223024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.195:50189)
0808 02:21:40.223744 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:40.223822 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0808 02:21:40.226104 (+ 2282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:21:40.227171 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:21:40.227181 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:21:40.227656 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:21:40.228398 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:40.228418 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.229506 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.229511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:40.230609 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:40.230624 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.231329 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.231333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:40.231575 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:40.232567 (+ 992us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:21:40.232592 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:21:40.234642 (+ 2050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:21:40.236978 (+ 2336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:21:40.236985 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:21:40.237005 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:21:40.237370 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:21:40.237853 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:21:40.237856 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:21:40.237858 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:21:40.238079 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:21:40.238816 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:21:40.238824 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:21:40.239222 (+ 398us) client_negotiation.cc:769] Sending connection context
0808 02:21:40.239455 (+ 233us) client_negotiation.cc:240] Negotiation successful
0808 02:21:40.239836 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":210,"threads_started":1}
I20260808 02:21:40.242494 16321 heartbeater.cc:347] Connected to a master server at 127.24.79.254:38659
I20260808 02:21:40.242841 16321 heartbeater.cc:464] Registering TS with master...
I20260808 02:21:40.243547 16321 heartbeater.cc:511] Master 127.24.79.254:38659 requested a full tablet report, sending...
I20260808 02:21:40.244997 15663 ts_manager.cc:194] Registered new tserver with Master: d25af570d8a54b55879a780d3d5f3505 (127.24.79.195:38035)
I20260808 02:21:40.245692 15663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.195@KRBTEST.COM'} at 127.24.79.195:50189
I20260808 02:21:40.249075 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:21:40.254073 16326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:40.249916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60658 (local address 127.24.79.254:38659)
0808 02:21:40.250101 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:40.250105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:40.250322 (+ 217us) server_negotiation.cc:407] Connection header received
0808 02:21:40.250487 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:40.250491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:40.250618 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:40.250721 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:21:40.251212 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.251820 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.252822 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.253072 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.253169 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:40.253480 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:21:40.253651 (+ 171us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 02:21:40.253934 (+ 283us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60658: token expired
Metrics: {"server-negotiator.queue_time_us":81}
W20260808 02:21:40.254182 16326 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60658: token expired
W20260808 02:21:40.254666 15558 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.24.79.254:38659)
I20260808 02:21:40.263602 16326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:40.255133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60668 (local address 127.24.79.254:38659)
0808 02:21:40.255273 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:40.255277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:40.255338 (+ 61us) server_negotiation.cc:407] Connection header received
0808 02:21:40.255486 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:40.255489 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:40.255534 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:40.255604 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:21:40.256530 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.257068 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.257937 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.258169 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.258264 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:40.259901 (+ 1637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:21:40.259905 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:21:40.259908 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:21:40.259943 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:21:40.261684 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:40.262235 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:40.262239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:40.262241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:40.262338 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:21:40.262617 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:21:40.262620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:21:40.262622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:21:40.262862 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:21:40.263042 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 02:21:40.263294 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 02:21:40.263433 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 02:21:40.266954 15663 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60668:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:21:40.270864 15663 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:21:40.281193 15849 tablet_service.cc:1467] Processing CreateTablet for tablet ee51145368a841b5bd6360db9e2fd5f3 (DEFAULT_TABLE table=test-table [id=3a8fe8c967aa4f4eb0b8f7460c847339]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:21:40.282037 15849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee51145368a841b5bd6360db9e2fd5f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:21:40.285523 15961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:40.279721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:38035 (local address 127.0.0.1:58804)
0808 02:21:40.279852 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:40.279868 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:21:40.279953 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:21:40.281171 (+ 1218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:21:40.281174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:21:40.281195 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:21:40.281323 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:40.281330 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.282957 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.282961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:40.284019 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:40.284030 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.284944 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.284948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:40.285059 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:40.285132 (+ 73us) client_negotiation.cc:769] Sending connection context
0808 02:21:40.285247 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 02:21:40.285357 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":5}
I20260808 02:21:40.286365 16335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:40.280078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58804 (local address 127.24.79.195:38035)
0808 02:21:40.280536 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:40.280551 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:40.280578 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:21:40.280689 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:40.280698 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:40.280993 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:40.281236 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:21:40.281423 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.282780 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.284170 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.284810 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.284932 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:40.285116 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 02:21:40.286060 (+ 944us) server_negotiation.cc:299] Negotiation successful
0808 02:21:40.286203 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":214,"threads_started":1}
I20260808 02:21:40.286392 16340 tablet_bootstrap.cc:492] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2: Bootstrap starting.
I20260808 02:21:40.286641 16336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:40.280427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:44439 (local address 127.0.0.1:49184)
0808 02:21:40.280771 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:40.280785 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:21:40.280926 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:21:40.281923 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:21:40.281927 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:21:40.281997 (+ 70us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:21:40.282104 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:40.282111 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.283533 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.283537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:40.284678 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:40.284690 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.286115 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.286129 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:40.286236 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:40.286263 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 02:21:40.286398 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 02:21:40.286490 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":114,"threads_started":1}
I20260808 02:21:40.287278 16339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:40.280916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49184 (local address 127.24.79.194:44439)
0808 02:21:40.281392 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:40.281405 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:40.281433 (+ 28us) server_negotiation.cc:407] Connection header received
0808 02:21:40.281515 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:40.281521 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:40.281749 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:40.281912 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:21:40.282261 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.283390 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.284848 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.285750 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.285881 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:40.286044 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 02:21:40.286986 (+ 942us) server_negotiation.cc:299] Negotiation successful
0808 02:21:40.287116 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":142,"threads_started":1}
I20260808 02:21:40.287634 16340 tablet_bootstrap.cc:654] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2: Neither blocks nor log segments found. Creating new log.
I20260808 02:21:40.289094 16340 tablet_bootstrap.cc:492] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2: No bootstrap required, opened a new log
I20260808 02:21:40.289279 16340 ts_tablet_manager.cc:1397] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:21:40.290206 16340 raft_consensus.cc:348] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } } peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } }
I20260808 02:21:40.290357 16340 raft_consensus.cc:374] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:21:40.290403 16340 raft_consensus.cc:729] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df902d8757214a8da739a4da5255e2e2, State: Initialized, Role: FOLLOWER
I20260808 02:21:40.290627 16340 consensus_queue.cc:260] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } } peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } }
I20260808 02:21:40.290400 16049 tablet_service.cc:1467] Processing CreateTablet for tablet ee51145368a841b5bd6360db9e2fd5f3 (DEFAULT_TABLE table=test-table [id=3a8fe8c967aa4f4eb0b8f7460c847339]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:21:40.291070 16340 ts_tablet_manager.cc:1428] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:21:40.290405 16233 tablet_service.cc:1467] Processing CreateTablet for tablet ee51145368a841b5bd6360db9e2fd5f3 (DEFAULT_TABLE table=test-table [id=3a8fe8c967aa4f4eb0b8f7460c847339]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:21:40.291813 16049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee51145368a841b5bd6360db9e2fd5f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:21:40.292196 16233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee51145368a841b5bd6360db9e2fd5f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:21:40.300594 16344 tablet_bootstrap.cc:492] T ee51145368a841b5bd6360db9e2fd5f3 P d25af570d8a54b55879a780d3d5f3505: Bootstrap starting.
I20260808 02:21:40.300644 16345 tablet_bootstrap.cc:492] T ee51145368a841b5bd6360db9e2fd5f3 P eef8385f4f864c02a4e467439d492ef1: Bootstrap starting.
I20260808 02:21:40.303479 16344 tablet_bootstrap.cc:654] T ee51145368a841b5bd6360db9e2fd5f3 P d25af570d8a54b55879a780d3d5f3505: Neither blocks nor log segments found. Creating new log.
I20260808 02:21:40.303515 16345 tablet_bootstrap.cc:654] T ee51145368a841b5bd6360db9e2fd5f3 P eef8385f4f864c02a4e467439d492ef1: Neither blocks nor log segments found. Creating new log.
I20260808 02:21:40.304373 16345 log.cc:824] T ee51145368a841b5bd6360db9e2fd5f3 P eef8385f4f864c02a4e467439d492ef1: Log is configured to *not* fsync() on all Append() calls
I20260808 02:21:40.304373 16344 log.cc:824] T ee51145368a841b5bd6360db9e2fd5f3 P d25af570d8a54b55879a780d3d5f3505: Log is configured to *not* fsync() on all Append() calls
I20260808 02:21:40.306775 16345 tablet_bootstrap.cc:492] T ee51145368a841b5bd6360db9e2fd5f3 P eef8385f4f864c02a4e467439d492ef1: No bootstrap required, opened a new log
I20260808 02:21:40.306820 16344 tablet_bootstrap.cc:492] T ee51145368a841b5bd6360db9e2fd5f3 P d25af570d8a54b55879a780d3d5f3505: No bootstrap required, opened a new log
I20260808 02:21:40.307066 16344 ts_tablet_manager.cc:1397] T ee51145368a841b5bd6360db9e2fd5f3 P d25af570d8a54b55879a780d3d5f3505: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260808 02:21:40.307068 16345 ts_tablet_manager.cc:1397] T ee51145368a841b5bd6360db9e2fd5f3 P eef8385f4f864c02a4e467439d492ef1: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 02:21:40.310336 16345 raft_consensus.cc:348] T ee51145368a841b5bd6360db9e2fd5f3 P eef8385f4f864c02a4e467439d492ef1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } } peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } }
I20260808 02:21:40.310336 16344 raft_consensus.cc:348] T ee51145368a841b5bd6360db9e2fd5f3 P d25af570d8a54b55879a780d3d5f3505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } } peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } }
I20260808 02:21:40.310688 16345 raft_consensus.cc:374] T ee51145368a841b5bd6360db9e2fd5f3 P eef8385f4f864c02a4e467439d492ef1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:21:40.310688 16344 raft_consensus.cc:374] T ee51145368a841b5bd6360db9e2fd5f3 P d25af570d8a54b55879a780d3d5f3505 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:21:40.310746 16344 raft_consensus.cc:729] T ee51145368a841b5bd6360db9e2fd5f3 P d25af570d8a54b55879a780d3d5f3505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d25af570d8a54b55879a780d3d5f3505, State: Initialized, Role: FOLLOWER
I20260808 02:21:40.310756 16345 raft_consensus.cc:729] T ee51145368a841b5bd6360db9e2fd5f3 P eef8385f4f864c02a4e467439d492ef1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eef8385f4f864c02a4e467439d492ef1, State: Initialized, Role: FOLLOWER
I20260808 02:21:40.311345 16344 consensus_queue.cc:260] T ee51145368a841b5bd6360db9e2fd5f3 P d25af570d8a54b55879a780d3d5f3505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } } peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } }
I20260808 02:21:40.311345 16345 consensus_queue.cc:260] T ee51145368a841b5bd6360db9e2fd5f3 P eef8385f4f864c02a4e467439d492ef1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } } peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } }
I20260808 02:21:40.312335 16321 heartbeater.cc:503] Master 127.24.79.254:38659 was elected leader, sending a full tablet report...
I20260808 02:21:40.312522 16141 heartbeater.cc:503] Master 127.24.79.254:38659 was elected leader, sending a full tablet report...
I20260808 02:21:40.312644 16345 ts_tablet_manager.cc:1428] T ee51145368a841b5bd6360db9e2fd5f3 P eef8385f4f864c02a4e467439d492ef1: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 02:21:40.312654 16344 ts_tablet_manager.cc:1428] T ee51145368a841b5bd6360db9e2fd5f3 P d25af570d8a54b55879a780d3d5f3505: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260808 02:21:40.471558 16322 tablet.cc:2369] T ee51145368a841b5bd6360db9e2fd5f3 P d25af570d8a54b55879a780d3d5f3505: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:21:40.493738 15952 tablet.cc:2369] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:21:40.507829 16343 raft_consensus.cc:482] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:21:40.508127 16343 raft_consensus.cc:504] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } } peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } }
I20260808 02:21:40.509361 16343 leader_election.cc:288] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d25af570d8a54b55879a780d3d5f3505 (127.24.79.195:38035), eef8385f4f864c02a4e467439d492ef1 (127.24.79.194:44439)
W20260808 02:21:40.511221 16142 tablet.cc:2369] T ee51145368a841b5bd6360db9e2fd5f3 P eef8385f4f864c02a4e467439d492ef1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:21:40.514351 16335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:40.509902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:37563 (local address 127.24.79.195:38035)
0808 02:21:40.510035 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:40.510039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:40.510101 (+ 62us) server_negotiation.cc:407] Connection header received
0808 02:21:40.510334 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:40.510338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:40.510395 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:40.510500 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:21:40.510786 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.511509 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.512966 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.513597 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.513704 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:40.513754 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 02:21:40.514128 (+ 374us) server_negotiation.cc:299] Negotiation successful
0808 02:21:40.514216 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:21:40.514364 16352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:40.509705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:38035 (local address 127.24.79.193:37563)
0808 02:21:40.510013 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:40.510030 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:21:40.510180 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:21:40.510532 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:21:40.510536 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:21:40.510552 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:21:40.510661 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:40.510667 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.511650 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.511654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:40.512800 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:40.512812 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.513742 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.513747 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:40.513863 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:40.513933 (+ 70us) client_negotiation.cc:769] Sending connection context
0808 02:21:40.514027 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 02:21:40.514187 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":23,"thread_start_us":96,"threads_started":1}
I20260808 02:21:40.514955 16339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:40.510345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:51081 (local address 127.24.79.194:44439)
0808 02:21:40.510478 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:40.510483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:40.510497 (+ 14us) server_negotiation.cc:407] Connection header received
0808 02:21:40.510604 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:40.510607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:40.510696 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:40.510774 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:21:40.511478 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.512274 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.513603 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.514216 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.514341 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:40.514393 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 02:21:40.514707 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 02:21:40.514798 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:21:40.515324 16354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:40.510188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:44439 (local address 127.24.79.193:51081)
0808 02:21:40.510417 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:40.510427 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:21:40.510498 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:21:40.510799 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:21:40.510801 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:21:40.510817 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:21:40.510947 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:40.510954 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.512447 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.512451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:40.513431 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:21:40.513443 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.514374 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.514378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:21:40.514491 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:40.514517 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 02:21:40.515132 (+ 615us) client_negotiation.cc:240] Negotiation successful
0808 02:21:40.515214 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":1,"thread_start_us":62,"threads_started":1}
I20260808 02:21:40.515275 16260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee51145368a841b5bd6360db9e2fd5f3" candidate_uuid: "df902d8757214a8da739a4da5255e2e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d25af570d8a54b55879a780d3d5f3505" is_pre_election: true
I20260808 02:21:40.515671 16260 raft_consensus.cc:2454] T ee51145368a841b5bd6360db9e2fd5f3 P d25af570d8a54b55879a780d3d5f3505 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df902d8757214a8da739a4da5255e2e2 in term 0.
I20260808 02:21:40.516026 16077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee51145368a841b5bd6360db9e2fd5f3" candidate_uuid: "df902d8757214a8da739a4da5255e2e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eef8385f4f864c02a4e467439d492ef1" is_pre_election: true
I20260808 02:21:40.516250 15778 leader_election.cc:302] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d25af570d8a54b55879a780d3d5f3505, df902d8757214a8da739a4da5255e2e2; no voters:
I20260808 02:21:40.516415 16077 raft_consensus.cc:2454] T ee51145368a841b5bd6360db9e2fd5f3 P eef8385f4f864c02a4e467439d492ef1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df902d8757214a8da739a4da5255e2e2 in term 0.
I20260808 02:21:40.516614 16343 raft_consensus.cc:2781] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:21:40.516717 16343 raft_consensus.cc:482] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:21:40.516752 16343 raft_consensus.cc:3037] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:21:40.517925 16343 raft_consensus.cc:504] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } } peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } }
I20260808 02:21:40.518317 16343 leader_election.cc:288] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 1 election: Requested vote from peers d25af570d8a54b55879a780d3d5f3505 (127.24.79.195:38035), eef8385f4f864c02a4e467439d492ef1 (127.24.79.194:44439)
I20260808 02:21:40.518843 16260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee51145368a841b5bd6360db9e2fd5f3" candidate_uuid: "df902d8757214a8da739a4da5255e2e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d25af570d8a54b55879a780d3d5f3505"
I20260808 02:21:40.518842 16077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee51145368a841b5bd6360db9e2fd5f3" candidate_uuid: "df902d8757214a8da739a4da5255e2e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eef8385f4f864c02a4e467439d492ef1"
I20260808 02:21:40.519027 16077 raft_consensus.cc:3037] T ee51145368a841b5bd6360db9e2fd5f3 P eef8385f4f864c02a4e467439d492ef1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:21:40.519027 16260 raft_consensus.cc:3037] T ee51145368a841b5bd6360db9e2fd5f3 P d25af570d8a54b55879a780d3d5f3505 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:21:40.520519 16260 raft_consensus.cc:2454] T ee51145368a841b5bd6360db9e2fd5f3 P d25af570d8a54b55879a780d3d5f3505 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df902d8757214a8da739a4da5255e2e2 in term 1.
I20260808 02:21:40.520519 16077 raft_consensus.cc:2454] T ee51145368a841b5bd6360db9e2fd5f3 P eef8385f4f864c02a4e467439d492ef1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df902d8757214a8da739a4da5255e2e2 in term 1.
I20260808 02:21:40.520957 15778 leader_election.cc:302] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: df902d8757214a8da739a4da5255e2e2, eef8385f4f864c02a4e467439d492ef1; no voters:
I20260808 02:21:40.521204 16343 raft_consensus.cc:2781] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:21:40.521481 16343 raft_consensus.cc:686] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 1 LEADER]: Becoming Leader. State: Replica: df902d8757214a8da739a4da5255e2e2, State: Running, Role: LEADER
I20260808 02:21:40.521793 16343 consensus_queue.cc:237] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } } peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } }
I20260808 02:21:40.525547 15663 catalog_manager.cc:5495] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to df902d8757214a8da739a4da5255e2e2 (127.24.79.193). New cstate: current_term: 1 leader_uuid: "df902d8757214a8da739a4da5255e2e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } health_report { overall_health: HEALTHY } } }
I20260808 02:21:40.593219 16358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:40.588729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35246 (local address 127.24.79.193:41277)
0808 02:21:40.589201 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0808 02:21:40.589208 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:21:40.589224 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:21:40.589321 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:21:40.589323 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:21:40.589391 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:21:40.589538 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:21:40.589838 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.590369 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.591324 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:21:40.591562 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:21:40.591668 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:21:40.591973 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:21:40.592504 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:21:40.592636 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 02:21:40.592726 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 02:21:40.592799 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":166,"threads_started":1}
I20260808 02:21:40.598325 16260 raft_consensus.cc:1261] T ee51145368a841b5bd6360db9e2fd5f3 P d25af570d8a54b55879a780d3d5f3505 [term 1 FOLLOWER]: Refusing update from remote peer df902d8757214a8da739a4da5255e2e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:21:40.598321 16077 raft_consensus.cc:1261] T ee51145368a841b5bd6360db9e2fd5f3 P eef8385f4f864c02a4e467439d492ef1 [term 1 FOLLOWER]: Refusing update from remote peer df902d8757214a8da739a4da5255e2e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:21:40.599246 16355 consensus_queue.cc:1035] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:21:40.599468 16343 consensus_queue.cc:1035] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:21:40.608826 16362 mvcc.cc:205] Tried to move back new op lower bound from 7316093749642506240 to 7316093749340299264. Current Snapshot: MvccSnapshot[applied={T|T < 7316093749642506240}]
I20260808 02:21:40.618969 15663 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:21:40.619199 15663 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:21:40.622107 15663 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0: Sending DeleteTablet for 3 replicas of tablet ee51145368a841b5bd6360db9e2fd5f3
I20260808 02:21:40.623032 15849 tablet_service.cc:1514] Processing DeleteTablet for tablet ee51145368a841b5bd6360db9e2fd5f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:21:40 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:35238
I20260808 02:21:40.623198 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 15723
I20260808 02:21:40.625767 16049 tablet_service.cc:1514] Processing DeleteTablet for tablet ee51145368a841b5bd6360db9e2fd5f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:21:40 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:49184
I20260808 02:21:40.632094 16233 tablet_service.cc:1514] Processing DeleteTablet for tablet ee51145368a841b5bd6360db9e2fd5f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:21:40 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:58804
I20260808 02:21:40.633785 16373 tablet_replica.cc:330] T ee51145368a841b5bd6360db9e2fd5f3 P eef8385f4f864c02a4e467439d492ef1: stopping tablet replica
I20260808 02:21:40.634599 16373 raft_consensus.cc:2229] T ee51145368a841b5bd6360db9e2fd5f3 P eef8385f4f864c02a4e467439d492ef1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:21:40.635257 16376 tablet_replica.cc:330] T ee51145368a841b5bd6360db9e2fd5f3 P d25af570d8a54b55879a780d3d5f3505: stopping tablet replica
I20260808 02:21:40.635318 16373 raft_consensus.cc:2258] T ee51145368a841b5bd6360db9e2fd5f3 P eef8385f4f864c02a4e467439d492ef1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:21:40.635686 16376 raft_consensus.cc:2229] T ee51145368a841b5bd6360db9e2fd5f3 P d25af570d8a54b55879a780d3d5f3505 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:21:40.636070 16376 raft_consensus.cc:2258] T ee51145368a841b5bd6360db9e2fd5f3 P d25af570d8a54b55879a780d3d5f3505 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:21:40.637429 16373 ts_tablet_manager.cc:1905] T ee51145368a841b5bd6360db9e2fd5f3 P eef8385f4f864c02a4e467439d492ef1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:21:40.637590 16376 ts_tablet_manager.cc:1905] T ee51145368a841b5bd6360db9e2fd5f3 P d25af570d8a54b55879a780d3d5f3505: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:21:40.640664 16376 ts_tablet_manager.cc:1918] T ee51145368a841b5bd6360db9e2fd5f3 P d25af570d8a54b55879a780d3d5f3505: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:21:40.640664 16373 ts_tablet_manager.cc:1918] T ee51145368a841b5bd6360db9e2fd5f3 P eef8385f4f864c02a4e467439d492ef1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:21:40.640821 16373 log.cc:1196] T ee51145368a841b5bd6360db9e2fd5f3 P eef8385f4f864c02a4e467439d492ef1: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/ee51145368a841b5bd6360db9e2fd5f3
I20260808 02:21:40.640821 16376 log.cc:1196] T ee51145368a841b5bd6360db9e2fd5f3 P d25af570d8a54b55879a780d3d5f3505: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/ee51145368a841b5bd6360db9e2fd5f3
I20260808 02:21:40.641188 16376 ts_tablet_manager.cc:1939] T ee51145368a841b5bd6360db9e2fd5f3 P d25af570d8a54b55879a780d3d5f3505: Deleting consensus metadata
I20260808 02:21:40.641187 16373 ts_tablet_manager.cc:1939] T ee51145368a841b5bd6360db9e2fd5f3 P eef8385f4f864c02a4e467439d492ef1: Deleting consensus metadata
W20260808 02:21:40.642997 15642 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.79.193:41277 (error 108)
I20260808 02:21:40.643373 15645 catalog_manager.cc:4841] TS d25af570d8a54b55879a780d3d5f3505 (127.24.79.195:38035): tablet ee51145368a841b5bd6360db9e2fd5f3 (table test-table [id=3a8fe8c967aa4f4eb0b8f7460c847339]) successfully deleted
I20260808 02:21:40.643594 15645 catalog_manager.cc:4841] TS eef8385f4f864c02a4e467439d492ef1 (127.24.79.194:44439): tablet ee51145368a841b5bd6360db9e2fd5f3 (table test-table [id=3a8fe8c967aa4f4eb0b8f7460c847339]) successfully deleted
I20260808 02:21:40.644073 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 15956
I20260808 02:21:40.652122 16336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:21:40.644167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:41277 (local address 127.0.0.1:35258)
0808 02:21:40.651833 (+ 7666us) negotiation.cc:107] Waiting for socket to connect
0808 02:21:40.651982 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.193:41277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7499}
W20260808 02:21:40.652419 15642 catalog_manager.cc:4584] TS df902d8757214a8da739a4da5255e2e2 (127.24.79.193:41277): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ee51145368a841b5bd6360db9e2fd5f3: Network error: Client connection negotiation failed: client connection to 127.24.79.193:41277: connect: Connection refused (error 111)
I20260808 02:21:40.662364 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 16143
I20260808 02:21:40.694736 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 15615
I20260808 02:22:00.707325 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:38659
--webserver_interface=127.24.79.254
--webserver_port=42817
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:40593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:38659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:00.838435 16380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:00.838806 16380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:00.838871 16380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:00.843282 16380 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 02:22:00.843372 16380 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 02:22:00.843394 16380 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 02:22:00.843413 16380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:00.843438 16380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:00.843462 16380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:00.843485 16380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:00.843515 16380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:00.849644 16380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:38659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:38659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=42817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:00.851029 16380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:00.852221 16380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:00.862895 16389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:00.862967 16387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:00.863119 16380 server_base.cc:1034] running on GCE node
W20260808 02:22:00.863027 16386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:00.863927 16380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:00.865306 16380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:00.866562 16380 hybrid_clock.cc:648] HybridClock initialized: now 1786155720866537 us; error 43 us; skew 500 ppm
Aug 08 02:22:00 dist-test-slave-1vmg krb5kdc[14862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155720, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:00.870409 16380 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:22:00.871814 16380 webserver.cc:460] Webserver started at http://127.24.79.254:42817/ using document root <none> and password file <none>
I20260808 02:22:00.872555 16380 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:00.872646 16380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:00.877594 16380 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 02:22:00.880436 16397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:00.882099 16380 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:22:00.882248 16380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "69b642c340c5439dafce5fb8caefdce0"
format_stamp: "Formatted at 2026-08-08 02:21:17 on dist-test-slave-1vmg"
server_key: "67108216d134b7ee5cfcd1d43a781919"
server_key_iv: "b0ea4e4b14fb21ecbbe9305acb627156"
server_key_version: "encryptionkey@0"
I20260808 02:22:00.882975 16380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:00.896817 16380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:00.900637 16380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:00.900862 16380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:00.909214 16380 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:38659
I20260808 02:22:00.909523 16469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:38659 every 8 connection(s)
I20260808 02:22:00.910468 16380 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:00.915482 16471 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:22:00.917380 16471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0: Bootstrap starting.
I20260808 02:22:00.917830 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 16380
I20260808 02:22:00.918355 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:41277
--local_ip_for_outbound_sockets=127.24.79.193
--tserver_master_addrs=127.24.79.254:38659
--webserver_port=36793
--webserver_interface=127.24.79.193
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:40593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:22:00.938961 16471 log.cc:824] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:00.951674 16471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:22:00.952104 16471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0: Bootstrap complete.
I20260808 02:22:00.956539 16471 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b642c340c5439dafce5fb8caefdce0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38659 } }
I20260808 02:22:00.956983 16471 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69b642c340c5439dafce5fb8caefdce0, State: Initialized, Role: FOLLOWER
I20260808 02:22:00.957558 16471 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b642c340c5439dafce5fb8caefdce0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38659 } }
I20260808 02:22:00.957695 16471 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:00.957765 16471 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:00.957872 16471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 2 FOLLOWER]: Advancing to term 3
I20260808 02:22:00.960151 16471 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b642c340c5439dafce5fb8caefdce0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38659 } }
I20260808 02:22:00.960530 16471 leader_election.cc:302] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69b642c340c5439dafce5fb8caefdce0; no voters:
I20260808 02:22:00.960881 16471 leader_election.cc:288] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 02:22:00.961036 16476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 02:22:00.961336 16476 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [term 3 LEADER]: Becoming Leader. State: Replica: 69b642c340c5439dafce5fb8caefdce0, State: Running, Role: LEADER
I20260808 02:22:00.961750 16476 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b642c340c5439dafce5fb8caefdce0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38659 } }
I20260808 02:22:00.962489 16471 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:00.963392 16477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "69b642c340c5439dafce5fb8caefdce0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b642c340c5439dafce5fb8caefdce0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38659 } } }
I20260808 02:22:00.963611 16477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:00.962858 16476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69b642c340c5439dafce5fb8caefdce0. Latest consensus state: current_term: 3 leader_uuid: "69b642c340c5439dafce5fb8caefdce0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b642c340c5439dafce5fb8caefdce0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38659 } } }
I20260808 02:22:00.964438 16476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:00.968559 16497 catalog_manager.cc:1259] Loaded cluster ID: 071d8474e0dd413bb7ec01c0edfdb77b
I20260808 02:22:00.968669 16497 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0: loading cluster ID for follower catalog manager: success
I20260808 02:22:00.971598 16497 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 69b642c340c5439dafce5fb8caefdce0: acquiring CA information for follower catalog manager: success
F20260808 02:22:00.973059 16497 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f04e88059ad google::LogMessage::Fail() at ??:0
@ 0x7f04e880888a google::LogMessage::SendToLog() at ??:0
@ 0x7f04e88054de google::LogMessage::Flush() at ??:0
@ 0x7f04e8806a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f04eb4c918d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f04f316e989 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7f04f314b023 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7f04f314b54b kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7f04f3144652 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7f04f31439e9 _ZZN4kudu6master21CatalogManagerBgTasks4InitEvENKUlvE_clEv at ??:0
@ 0x7f04f318a541 _ZNSt17_Function_handlerIFvvEZN4kudu6master21CatalogManagerBgTasks4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f04eaa48512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f04ec08c6db start_thread at ??:0
@ 0x7f04e730361f clone at ??:0
W20260808 02:22:01.087855 16473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:01.088328 16473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:01.088405 16473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:01.095016 16473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:01.095125 16473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:01.095222 16473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:22:01.102329 16473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:41277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=36793
--tserver_master_addrs=127.24.79.254:38659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:01.103739 16473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:01.105131 16473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:01.117802 16529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:01.117935 16527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:01.118095 16473 server_base.cc:1034] running on GCE node
W20260808 02:22:01.117802 16526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:01.118906 16473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:01.120041 16473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:01.121284 16473 hybrid_clock.cc:648] HybridClock initialized: now 1786155721121257 us; error 44 us; skew 500 ppm
Aug 08 02:22:01 dist-test-slave-1vmg krb5kdc[14862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155721, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:01.125558 16473 init.cc:377] Logged in from keytab as kudu/127.24.79.193@KRBTEST.COM (short username kudu)
I20260808 02:22:01.127146 16473 webserver.cc:460] Webserver started at http://127.24.79.193:36793/ using document root <none> and password file <none>
I20260808 02:22:01.128093 16473 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:01.128387 16473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:01.134882 16473 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260808 02:22:01.139058 16541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:01.140961 16473 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260808 02:22:01.141134 16473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "df902d8757214a8da739a4da5255e2e2"
format_stamp: "Formatted at 2026-08-08 02:21:18 on dist-test-slave-1vmg"
server_key: "ddf9554af2f50e833f96134e1208266e"
server_key_iv: "8f1234e252ef67825aacaa24fcb4551f"
server_key_version: "encryptionkey@0"
I20260808 02:22:01.141815 16473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:01.177595 16473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:01.181854 16473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:01.182133 16473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:01.182858 16473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:22:01.184321 16552 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 02:22:01.190968 16473 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 02:22:01.191095 16473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.001s
I20260808 02:22:01.191248 16473 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 02:22:01.193362 16473 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 02:22:01.193434 16473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 02:22:01.193857 16552 tablet_bootstrap.cc:492] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2: Bootstrap starting.
I20260808 02:22:01.204960 16473 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:41277
I20260808 02:22:01.205344 16703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:41277 every 8 connection(s)
I20260808 02:22:01.206266 16473 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:22:01.209789 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 16473
I20260808 02:22:01.210304 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:44439
--local_ip_for_outbound_sockets=127.24.79.194
--tserver_master_addrs=127.24.79.254:38659
--webserver_port=38163
--webserver_interface=127.24.79.194
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:40593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:22:01.218478 16552 log.cc:824] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:01.230356 16552 tablet_bootstrap.cc:492] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:22:01.230832 16552 tablet_bootstrap.cc:492] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2: Bootstrap complete.
I20260808 02:22:01.232092 16552 ts_tablet_manager.cc:1397] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2: Time spent bootstrapping tablet: real 0.039s user 0.027s sys 0.009s
I20260808 02:22:01.235152 16552 raft_consensus.cc:348] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } } peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } }
I20260808 02:22:01.235684 16552 raft_consensus.cc:729] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: df902d8757214a8da739a4da5255e2e2, State: Initialized, Role: FOLLOWER
I20260808 02:22:01.236282 16552 consensus_queue.cc:260] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } } peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } }
I20260808 02:22:01.237469 16552 ts_tablet_manager.cc:1428] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260808 02:22:01.238754 16708 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:22:01.208632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.193:55803)
0808 02:22:01.209239 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:01.209306 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:01.211234 (+ 1928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:01.237916 (+ 26682us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":165,"threads_started":1}
W20260808 02:22:01.239476 16704 heartbeater.cc:644] Failed to heartbeat to 127.24.79.254:38659 (0 consecutive failures): Network error: Failed to ping master at 127.24.79.254:38659: Client connection negotiation failed: client connection to 127.24.79.254:38659: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 02:22:01.240453 16708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:01.240049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.193:47471)
0808 02:22:01.240291 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:01.240377 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260808 02:22:01.241375 16708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:01.240968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.193:46837)
0808 02:22:01.241235 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:01.241304 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260808 02:22:01.241581 16704 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:22:01.351972 16709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:01.352339 16709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:01.352411 16709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:01.356928 16709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:01.357041 16709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:01.357132 16709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:22:01.363958 16709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:44439
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=38163
--tserver_master_addrs=127.24.79.254:38659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:01.365470 16709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:01.366806 16709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:01.378649 16720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:01.378624 16717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:01.378629 16723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:01.379195 16709 server_base.cc:1034] running on GCE node
I20260808 02:22:01.379938 16709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:01.380762 16709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:01.381987 16709 hybrid_clock.cc:648] HybridClock initialized: now 1786155721381927 us; error 76 us; skew 500 ppm
Aug 08 02:22:01 dist-test-slave-1vmg krb5kdc[14862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155721, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:01.385552 16709 init.cc:377] Logged in from keytab as kudu/127.24.79.194@KRBTEST.COM (short username kudu)
I20260808 02:22:01.386703 16709 webserver.cc:460] Webserver started at http://127.24.79.194:38163/ using document root <none> and password file <none>
I20260808 02:22:01.387393 16709 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:01.387480 16709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:01.391865 16709 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:22:01.394280 16734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:01.396004 16709 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:22:01.396183 16709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "eef8385f4f864c02a4e467439d492ef1"
format_stamp: "Formatted at 2026-08-08 02:21:18 on dist-test-slave-1vmg"
server_key: "0983995ffa15abd31666070d87bbab2e"
server_key_iv: "c2119a443dc760383193fba777d0f6d4"
server_key_version: "encryptionkey@0"
I20260808 02:22:01.396831 16709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:01.427425 16709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:01.431594 16709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:01.431833 16709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:01.432564 16709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:22:01.433629 16709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:22:01.433686 16709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:01.433748 16709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:22:01.433776 16709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:01.444932 16709 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:44439
I20260808 02:22:01.444953 16888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:44439 every 8 connection(s)
I20260808 02:22:01.447117 16709 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:22:01.451977 16891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:01.449797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.194:45395)
0808 02:22:01.450417 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:01.451000 (+ 583us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":150,"threads_started":1}
I20260808 02:22:01.452227 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 16709
I20260808 02:22:01.452739 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:38035
--local_ip_for_outbound_sockets=127.24.79.195
--tserver_master_addrs=127.24.79.254:38659
--webserver_port=41057
--webserver_interface=127.24.79.195
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:40593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:01.452903 16889 heartbeater.cc:644] Failed to heartbeat to 127.24.79.254:38659 (0 consecutive failures): Network error: Failed to ping master at 127.24.79.254:38659: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
I20260808 02:22:01.453761 16891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:01.453425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.194:50053)
0808 02:22:01.453595 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:01.453672 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 02:22:01.454628 16891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:01.454354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.194:56901)
0808 02:22:01.454488 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:01.454549 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 02:22:01.454898 16889 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:22:01.588024 16892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:01.588374 16892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:01.588423 16892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:01.592870 16892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:01.592957 16892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:01.593022 16892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:22:01.599054 16892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:38035
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=41057
--tserver_master_addrs=127.24.79.254:38659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:01.600383 16892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:01.601687 16892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:01.613698 16899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:01.613698 16904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:01.613710 16900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:01.613811 16892 server_base.cc:1034] running on GCE node
I20260808 02:22:01.614604 16892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:01.615501 16892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:01.616739 16892 hybrid_clock.cc:648] HybridClock initialized: now 1786155721616710 us; error 45 us; skew 500 ppm
Aug 08 02:22:01 dist-test-slave-1vmg krb5kdc[14862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155721, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:01.620349 16892 init.cc:377] Logged in from keytab as kudu/127.24.79.195@KRBTEST.COM (short username kudu)
I20260808 02:22:01.621595 16892 webserver.cc:460] Webserver started at http://127.24.79.195:41057/ using document root <none> and password file <none>
I20260808 02:22:01.622326 16892 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:01.622416 16892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:01.627341 16892 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:22:01.630379 16914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:01.632292 16892 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:22:01.632459 16892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "d25af570d8a54b55879a780d3d5f3505"
format_stamp: "Formatted at 2026-08-08 02:21:18 on dist-test-slave-1vmg"
server_key: "ba6463ad2a28384e0bf7c6fed0ce324e"
server_key_iv: "067dad33f8b8e8336205fd7cc11daed6"
server_key_version: "encryptionkey@0"
I20260808 02:22:01.633140 16892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:01.658041 16892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:01.662055 16892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:01.662356 16892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:01.663147 16892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:22:01.664170 16892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:22:01.664225 16892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:01.664286 16892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:22:01.664315 16892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:01.676152 16892 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:38035
I20260808 02:22:01.676268 17070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:38035 every 8 connection(s)
I20260808 02:22:01.677450 16892 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:22:01.682466 17073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:01.680392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.195:46927)
0808 02:22:01.681075 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:01.681582 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":179,"threads_started":1}
W20260808 02:22:01.683223 17071 heartbeater.cc:644] Failed to heartbeat to 127.24.79.254:38659 (0 consecutive failures): Network error: Failed to ping master at 127.24.79.254:38659: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
I20260808 02:22:01.683565 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 16892
I20260808 02:22:01.684192 17073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:01.683814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.195:60507)
0808 02:22:01.684024 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:01.684095 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260808 02:22:01.684892 17073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:01.684656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.195:51767)
0808 02:22:01.684784 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:01.684831 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 02:22:01.685042 17071 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 02:22:02.243163 17075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:02.242412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.193:55371)
0808 02:22:02.242877 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:02.243006 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":98,"threads_started":1}
I20260808 02:22:02.456286 17076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:02.455732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.194:53887)
0808 02:22:02.456062 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:02.456135 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":90,"threads_started":1}
I20260808 02:22:02.686540 17077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:02.685862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.195:60441)
0808 02:22:02.686301 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:02.686394 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":85,"threads_started":1}
I20260808 02:22:03.034039 17079 raft_consensus.cc:482] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:22:03.034292 17079 raft_consensus.cc:504] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } } peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } }
I20260808 02:22:03.035650 17079 leader_election.cc:288] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d25af570d8a54b55879a780d3d5f3505 (127.24.79.195:38035), eef8385f4f864c02a4e467439d492ef1 (127.24.79.194:44439)
Aug 08 02:22:03 dist-test-slave-1vmg krb5kdc[14862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155721, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.194@KRBTEST.COM
Aug 08 02:22:03 dist-test-slave-1vmg krb5kdc[14862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155721, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.195@KRBTEST.COM
I20260808 02:22:03.052325 17082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:03.036504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:44439 (local address 127.24.79.193:43353)
0808 02:22:03.036829 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:03.036842 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:03.036991 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:03.039034 (+ 2043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:03.039036 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:03.039334 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:03.039860 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:03.039879 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:03.041268 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:03.041271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:03.042409 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:03.042425 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:03.043280 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:03.043284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:03.043559 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:03.044283 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:22:03.044295 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:03.045999 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:03.048748 (+ 2749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:03.048755 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:03.048771 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:03.049145 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:03.049648 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:03.049652 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:03.049654 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:03.049929 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:03.050895 (+ 966us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:03.050904 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:03.051364 (+ 460us) client_negotiation.cc:769] Sending connection context
0808 02:22:03.051623 (+ 259us) client_negotiation.cc:240] Negotiation successful
0808 02:22:03.052148 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":530,"spinlock_wait_cycles":21120,"thread_start_us":85,"threads_started":1}
I20260808 02:22:03.052417 17084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:03.036724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:43353 (local address 127.24.79.194:44439)
0808 02:22:03.037136 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:03.037147 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:03.037180 (+ 33us) server_negotiation.cc:407] Connection header received
0808 02:22:03.037920 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:03.037928 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:03.038317 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:03.038613 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:03.040071 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:03.041087 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:03.042574 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:03.043142 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:03.043315 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:03.046310 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:03.046319 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:03.046340 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:03.046381 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:03.048617 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:03.049302 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:03.049310 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:03.049316 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:03.049501 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:03.050057 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:03.050061 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:03.050063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:03.050744 (+ 681us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:03.050884 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:03.051855 (+ 971us) server_negotiation.cc:299] Negotiation successful
0808 02:22:03.052211 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":111,"threads_started":1}
I20260808 02:22:03.052431 17081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:03.036254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:40089 (local address 127.24.79.195:38035)
0808 02:22:03.036734 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:03.036767 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:03.036822 (+ 55us) server_negotiation.cc:407] Connection header received
0808 02:22:03.037867 (+ 1045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:03.037875 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:03.038316 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:03.038607 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:03.040061 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:03.041091 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:03.042579 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:03.043140 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:03.043315 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:03.046648 (+ 3333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:03.046653 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:03.046666 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:03.046701 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:03.048840 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:03.049371 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:03.049377 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:03.049382 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:03.049522 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:03.050057 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:03.050061 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:03.050063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:03.050720 (+ 657us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:03.050884 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:03.051855 (+ 971us) server_negotiation.cc:299] Negotiation successful
0808 02:22:03.052243 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":67,"threads_started":1}
I20260808 02:22:03.052325 17080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:03.036022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:38035 (local address 127.24.79.193:40089)
0808 02:22:03.036363 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:03.036380 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:03.036801 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:03.038777 (+ 1976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:03.038785 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:03.039344 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:03.039861 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:03.039879 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:03.041268 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:03.041271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:03.042408 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:03.042425 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:03.043280 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:03.043284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:03.043595 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:03.044270 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:22:03.044287 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:03.046400 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:03.048953 (+ 2553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:03.048956 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:03.048958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:03.049257 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:03.049675 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:03.049677 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:03.049679 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:03.049929 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:03.050895 (+ 966us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:03.050904 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:03.051352 (+ 448us) client_negotiation.cc:769] Sending connection context
0808 02:22:03.051631 (+ 279us) client_negotiation.cc:240] Negotiation successful
0808 02:22:03.052067 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":259,"spinlock_wait_cycles":21632,"thread_start_us":120,"threads_started":1}
W20260808 02:22:03.055281 16547 leader_election.cc:341] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eef8385f4f864c02a4e467439d492ef1 (127.24.79.194:44439): Not found: Tablet not found: ee51145368a841b5bd6360db9e2fd5f3
W20260808 02:22:03.055584 16547 leader_election.cc:341] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d25af570d8a54b55879a780d3d5f3505 (127.24.79.195:38035): Not found: Tablet not found: ee51145368a841b5bd6360db9e2fd5f3
I20260808 02:22:03.055665 16547 leader_election.cc:302] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df902d8757214a8da739a4da5255e2e2; no voters: d25af570d8a54b55879a780d3d5f3505, eef8385f4f864c02a4e467439d492ef1
I20260808 02:22:03.055934 17079 raft_consensus.cc:2726] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:22:03.244558 17080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:03.244166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.193:49113)
0808 02:22:03.244385 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:03.244457 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260808 02:22:03.457830 17085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:03.457218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.194:44547)
0808 02:22:03.457614 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:03.457684 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":134,"threads_started":1}
I20260808 02:22:03.688273 17087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:03.687694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.195:57921)
0808 02:22:03.688025 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:03.688156 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":90,"threads_started":1}
I20260808 02:22:04.246321 17088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:04.245756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.193:42913)
0808 02:22:04.246088 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:04.246162 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":95,"threads_started":1}
I20260808 02:22:04.459481 17089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:04.458882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.194:36625)
0808 02:22:04.459299 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:04.459372 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":120,"threads_started":1}
I20260808 02:22:04.689893 17091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:04.689347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.195:46785)
0808 02:22:04.689706 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:04.689781 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":113,"threads_started":1}
I20260808 02:22:04.716377 17092 raft_consensus.cc:482] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:22:04.716531 17092 raft_consensus.cc:504] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } } peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } }
I20260808 02:22:04.717000 17092 leader_election.cc:288] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d25af570d8a54b55879a780d3d5f3505 (127.24.79.195:38035), eef8385f4f864c02a4e467439d492ef1 (127.24.79.194:44439)
W20260808 02:22:04.718127 16547 leader_election.cc:341] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eef8385f4f864c02a4e467439d492ef1 (127.24.79.194:44439): Not found: Tablet not found: ee51145368a841b5bd6360db9e2fd5f3
W20260808 02:22:04.718300 16547 leader_election.cc:341] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d25af570d8a54b55879a780d3d5f3505 (127.24.79.195:38035): Not found: Tablet not found: ee51145368a841b5bd6360db9e2fd5f3
I20260808 02:22:04.718341 16547 leader_election.cc:302] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df902d8757214a8da739a4da5255e2e2; no voters: d25af570d8a54b55879a780d3d5f3505, eef8385f4f864c02a4e467439d492ef1
I20260808 02:22:04.718544 17092 raft_consensus.cc:2726] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:22:05.247923 17094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:05.247361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.193:56813)
0808 02:22:05.247727 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:05.247809 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":90,"threads_started":1}
I20260808 02:22:05.461079 17095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:05.460524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.194:41971)
0808 02:22:05.460895 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:05.460968 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":99,"threads_started":1}
I20260808 02:22:05.691519 17101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:05.690861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.195:47701)
0808 02:22:05.691330 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:05.691408 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":113,"threads_started":1}
I20260808 02:22:06.249513 17102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:06.248964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.193:48509)
0808 02:22:06.249326 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:06.249394 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":89,"threads_started":1}
I20260808 02:22:06.462739 17105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:06.462264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.194:40907)
0808 02:22:06.462561 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:06.462631 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":84,"threads_started":1}
I20260808 02:22:06.477259 17106 raft_consensus.cc:482] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:22:06.477425 17106 raft_consensus.cc:504] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } } peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } }
I20260808 02:22:06.477861 17106 leader_election.cc:288] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d25af570d8a54b55879a780d3d5f3505 (127.24.79.195:38035), eef8385f4f864c02a4e467439d492ef1 (127.24.79.194:44439)
W20260808 02:22:06.478802 16547 leader_election.cc:341] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d25af570d8a54b55879a780d3d5f3505 (127.24.79.195:38035): Not found: Tablet not found: ee51145368a841b5bd6360db9e2fd5f3
W20260808 02:22:06.479000 16547 leader_election.cc:341] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eef8385f4f864c02a4e467439d492ef1 (127.24.79.194:44439): Not found: Tablet not found: ee51145368a841b5bd6360db9e2fd5f3
I20260808 02:22:06.479053 16547 leader_election.cc:302] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df902d8757214a8da739a4da5255e2e2; no voters: d25af570d8a54b55879a780d3d5f3505, eef8385f4f864c02a4e467439d492ef1
I20260808 02:22:06.479249 17106 raft_consensus.cc:2726] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:22:06.693214 17107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:06.692645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.195:57779)
0808 02:22:06.692983 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:06.693055 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":96,"threads_started":1}
I20260808 02:22:07.251303 17108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:07.250564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.193:50307)
0808 02:22:07.251055 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:07.251149 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":95,"threads_started":1}
I20260808 02:22:07.464401 17109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:07.463864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.194:59729)
0808 02:22:07.464213 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:07.464292 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":95,"threads_started":1}
I20260808 02:22:07.694846 17110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:07.694318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.195:48713)
0808 02:22:07.694640 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:07.694728 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":95,"threads_started":1}
I20260808 02:22:08.155560 17112 raft_consensus.cc:482] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:22:08.155742 17112 raft_consensus.cc:504] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } } peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } }
I20260808 02:22:08.156222 17112 leader_election.cc:288] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d25af570d8a54b55879a780d3d5f3505 (127.24.79.195:38035), eef8385f4f864c02a4e467439d492ef1 (127.24.79.194:44439)
W20260808 02:22:08.157210 16547 leader_election.cc:341] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eef8385f4f864c02a4e467439d492ef1 (127.24.79.194:44439): Not found: Tablet not found: ee51145368a841b5bd6360db9e2fd5f3
W20260808 02:22:08.157394 16547 leader_election.cc:341] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d25af570d8a54b55879a780d3d5f3505 (127.24.79.195:38035): Not found: Tablet not found: ee51145368a841b5bd6360db9e2fd5f3
I20260808 02:22:08.157451 16547 leader_election.cc:302] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df902d8757214a8da739a4da5255e2e2; no voters: d25af570d8a54b55879a780d3d5f3505, eef8385f4f864c02a4e467439d492ef1
I20260808 02:22:08.157657 17112 raft_consensus.cc:2726] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:22:08.252960 17113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:08.252428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.193:41405)
0808 02:22:08.252768 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:08.252850 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":87,"threads_started":1}
I20260808 02:22:08.466060 17114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:08.465431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.194:44459)
0808 02:22:08.465868 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:08.465938 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":133,"threads_started":1}
I20260808 02:22:08.696699 17115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:08.696048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.195:34283)
0808 02:22:08.696501 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:08.696584 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":101,"threads_started":1}
I20260808 02:22:09.254765 17117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:09.254146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.193:53199)
0808 02:22:09.254568 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:09.254649 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":144,"threads_started":1}
I20260808 02:22:09.467809 17118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:09.467267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.194:57519)
0808 02:22:09.467622 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:09.467695 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":108,"threads_started":1}
I20260808 02:22:09.698297 17119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:09.697774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.195:45591)
0808 02:22:09.698077 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:09.698150 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":139,"threads_started":1}
I20260808 02:22:10.256510 17121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:10.255927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.193:55847)
0808 02:22:10.256292 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:10.256375 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":105,"threads_started":1}
I20260808 02:22:10.469677 17122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:10.469019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.194:60397)
0808 02:22:10.469482 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:10.469561 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":165,"threads_started":1}
I20260808 02:22:10.699836 17123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:10.699277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.195:35519)
0808 02:22:10.699636 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:10.699708 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":90,"threads_started":1}
I20260808 02:22:11.258299 17125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:11.257738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.193:44951)
0808 02:22:11.258111 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:11.258187 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":94,"threads_started":1}
I20260808 02:22:11.471494 17126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:11.470881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.194:41015)
0808 02:22:11.471303 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:11.471384 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":92,"threads_started":1}
I20260808 02:22:11.701503 17127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:11.700944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.195:47901)
0808 02:22:11.701295 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:11.701370 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":93,"threads_started":1}
I20260808 02:22:12.084424 17128 raft_consensus.cc:482] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:22:12.084594 17128 raft_consensus.cc:504] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } } peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } }
I20260808 02:22:12.085124 17128 leader_election.cc:288] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d25af570d8a54b55879a780d3d5f3505 (127.24.79.195:38035), eef8385f4f864c02a4e467439d492ef1 (127.24.79.194:44439)
W20260808 02:22:12.086269 16547 leader_election.cc:341] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d25af570d8a54b55879a780d3d5f3505 (127.24.79.195:38035): Not found: Tablet not found: ee51145368a841b5bd6360db9e2fd5f3
W20260808 02:22:12.086419 16547 leader_election.cc:341] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eef8385f4f864c02a4e467439d492ef1 (127.24.79.194:44439): Not found: Tablet not found: ee51145368a841b5bd6360db9e2fd5f3
I20260808 02:22:12.086457 16547 leader_election.cc:302] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df902d8757214a8da739a4da5255e2e2; no voters: d25af570d8a54b55879a780d3d5f3505, eef8385f4f864c02a4e467439d492ef1
I20260808 02:22:12.086714 17128 raft_consensus.cc:2726] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:22:12.260030 17129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:12.259467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.193:52401)
0808 02:22:12.259835 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:12.259914 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":99,"threads_started":1}
I20260808 02:22:12.473201 17130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:12.472691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.194:56491)
0808 02:22:12.473005 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:12.473090 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":96,"threads_started":1}
I20260808 02:22:12.703373 17131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:12.702700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.195:49693)
0808 02:22:12.703173 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:12.703257 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":98,"threads_started":1}
I20260808 02:22:13.261786 17132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:13.261158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.193:45987)
0808 02:22:13.261582 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:13.261665 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":96,"threads_started":1}
I20260808 02:22:13.474882 17133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:13.474358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.194:37577)
0808 02:22:13.474694 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:13.474771 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":92,"threads_started":1}
I20260808 02:22:13.705133 17134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:13.704552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.195:43797)
0808 02:22:13.704918 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:13.704994 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":100,"threads_started":1}
I20260808 02:22:14.263557 17135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:14.262951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.193:34305)
0808 02:22:14.263375 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:14.263442 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":113,"threads_started":1}
I20260808 02:22:14.476600 17136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:14.476046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.194:54083)
0808 02:22:14.476414 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:14.476487 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":98,"threads_started":1}
I20260808 02:22:14.706838 17138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:14.706301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.195:54741)
0808 02:22:14.706666 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:14.706734 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":112,"threads_started":1}
I20260808 02:22:15.265265 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:15.264609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.193:36861)
0808 02:22:15.265017 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:15.265145 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":119,"threads_started":1}
I20260808 02:22:15.360613 17141 raft_consensus.cc:482] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:22:15.360770 17141 raft_consensus.cc:504] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25af570d8a54b55879a780d3d5f3505" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38035 } } peers { permanent_uuid: "eef8385f4f864c02a4e467439d492ef1" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 44439 } } peers { permanent_uuid: "df902d8757214a8da739a4da5255e2e2" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 41277 } }
I20260808 02:22:15.361268 17141 leader_election.cc:288] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d25af570d8a54b55879a780d3d5f3505 (127.24.79.195:38035), eef8385f4f864c02a4e467439d492ef1 (127.24.79.194:44439)
W20260808 02:22:15.362303 16547 leader_election.cc:341] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eef8385f4f864c02a4e467439d492ef1 (127.24.79.194:44439): Not found: Tablet not found: ee51145368a841b5bd6360db9e2fd5f3
W20260808 02:22:15.362457 16547 leader_election.cc:341] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d25af570d8a54b55879a780d3d5f3505 (127.24.79.195:38035): Not found: Tablet not found: ee51145368a841b5bd6360db9e2fd5f3
I20260808 02:22:15.362502 16547 leader_election.cc:302] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df902d8757214a8da739a4da5255e2e2; no voters: d25af570d8a54b55879a780d3d5f3505, eef8385f4f864c02a4e467439d492ef1
I20260808 02:22:15.362705 17141 raft_consensus.cc:2726] T ee51145368a841b5bd6360db9e2fd5f3 P df902d8757214a8da739a4da5255e2e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:22:15.478317 17142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:15.477675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.194:38313)
0808 02:22:15.478112 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:15.478195 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":120,"threads_started":1}
I20260808 02:22:15.708506 17143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:15.707924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.195:60643)
0808 02:22:15.708325 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:15.708395 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":133,"threads_started":1}
I20260808 02:22:16.267031 17144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:16.266482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.193:54223)
0808 02:22:16.266811 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:16.266883 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":85,"threads_started":1}
I20260808 02:22:16.479995 17145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:16.479479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.194:40441)
0808 02:22:16.479816 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:16.479884 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":90,"threads_started":1}
I20260808 02:22:16.692234 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 16473
I20260808 02:22:16.706254 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 16709
I20260808 02:22:16.719583 17146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:16.717224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38659 (local address 127.24.79.195:33585)
0808 02:22:16.719387 (+ 2163us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:16.719471 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.254:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2046,"thread_start_us":1660,"threads_started":1}
I20260808 02:22:16.725956 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 16892
2026-08-08T02:22:16Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (60429 ms)
[----------] 30 tests from SecurityITest (272066 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:22:16 dist-test-slave-1vmg krb5kdc[17149](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:22:16 dist-test-slave-1vmg krb5kdc[17149](info): set up 2 sockets
Aug 08 02:22:16 dist-test-slave-1vmg krb5kdc[17149](info): commencing operation
krb5kdc: starting...
W20260808 02:22:17.936225 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.007s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:22:18 dist-test-slave-1vmg krb5kdc[17149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:22:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:22:18.104030 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:40657
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:38783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:40657
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:18.234982 17168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:18.235332 17168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:18.235397 17168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:18.240159 17168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:18.240280 17168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:18.240314 17168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:18.240335 17168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:18.240355 17168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:18.246336 17168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:38783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:40657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:40657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:18.247900 17168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:18.249203 17168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:22:18.259979 17168 server_base.cc:1034] running on GCE node
W20260808 02:22:18.259909 17177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:18.259909 17176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:18.259909 17179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:18.261274 17168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:18.262991 17168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:18.264314 17168 hybrid_clock.cc:648] HybridClock initialized: now 1786155738264172 us; error 96 us; skew 500 ppm
Aug 08 02:22:18 dist-test-slave-1vmg krb5kdc[17149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155738, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:18.268074 17168 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:22:18.269429 17168 webserver.cc:460] Webserver started at http://127.24.79.254:44831/ using document root <none> and password file <none>
I20260808 02:22:18.270107 17168 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:18.270186 17168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:18.270448 17168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:18.272965 17168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "1f9cee260b5b45ab909619d7c28d8a2f"
format_stamp: "Formatted at 2026-08-08 02:22:18 on dist-test-slave-1vmg"
server_key: "4038c3cf27e2a57cbd2be770432f387e"
server_key_iv: "4abc288444f25476de2e1c5073973ee7"
server_key_version: "encryptionkey@0"
I20260808 02:22:18.273633 17168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "1f9cee260b5b45ab909619d7c28d8a2f"
format_stamp: "Formatted at 2026-08-08 02:22:18 on dist-test-slave-1vmg"
server_key: "4038c3cf27e2a57cbd2be770432f387e"
server_key_iv: "4abc288444f25476de2e1c5073973ee7"
server_key_version: "encryptionkey@0"
I20260808 02:22:18.278092 17168 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260808 02:22:18.281203 17188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:18.282763 17168 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:22:18.283015 17168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "1f9cee260b5b45ab909619d7c28d8a2f"
format_stamp: "Formatted at 2026-08-08 02:22:18 on dist-test-slave-1vmg"
server_key: "4038c3cf27e2a57cbd2be770432f387e"
server_key_iv: "4abc288444f25476de2e1c5073973ee7"
server_key_version: "encryptionkey@0"
I20260808 02:22:18.283176 17168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:18.295522 17168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:18.299830 17168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:18.300051 17168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:18.309237 17168 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:40657
I20260808 02:22:18.309365 17257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:40657 every 8 connection(s)
I20260808 02:22:18.310374 17168 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:18.313670 17259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:18.313997 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 17168
I20260808 02:22:18.314132 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:18.314472 24895 external_mini_cluster.cc:1428] Setting key 6a12e9e50dc88f569701cd5a69051254
I20260808 02:22:18.322227 17259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f9cee260b5b45ab909619d7c28d8a2f: Bootstrap starting.
I20260808 02:22:18.325340 17259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f9cee260b5b45ab909619d7c28d8a2f: Neither blocks nor log segments found. Creating new log.
Aug 08 02:22:18 dist-test-slave-1vmg krb5kdc[17149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:22:18.326479 17259 log.cc:824] T 00000000000000000000000000000000 P 1f9cee260b5b45ab909619d7c28d8a2f: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:18.329540 17259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f9cee260b5b45ab909619d7c28d8a2f: No bootstrap required, opened a new log
I20260808 02:22:18.332333 17263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:18.316210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60434 (local address 127.24.79.254:40657)
0808 02:22:18.316808 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:18.316831 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:18.316882 (+ 51us) server_negotiation.cc:407] Connection header received
0808 02:22:18.317823 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:18.317856 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:18.318375 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:18.318816 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:18.319715 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:18.321291 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:18.322214 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:18.322958 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:18.323133 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:18.326338 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:18.326345 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:18.326364 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:18.326403 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:18.328659 (+ 2256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:18.329338 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:18.329343 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:18.329350 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:18.329487 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:18.329840 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:18.329843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:18.329845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:18.330355 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:18.330537 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:18.330999 (+ 462us) server_negotiation.cc:299] Negotiation successful
0808 02:22:18.331480 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":169,"threads_started":1}
I20260808 02:22:18.333912 17259 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f9cee260b5b45ab909619d7c28d8a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f9cee260b5b45ab909619d7c28d8a2f" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 40657 } }
I20260808 02:22:18.334264 17259 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f9cee260b5b45ab909619d7c28d8a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:18.334334 17259 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f9cee260b5b45ab909619d7c28d8a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f9cee260b5b45ab909619d7c28d8a2f, State: Initialized, Role: FOLLOWER
I20260808 02:22:18.334872 17259 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f9cee260b5b45ab909619d7c28d8a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f9cee260b5b45ab909619d7c28d8a2f" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 40657 } }
I20260808 02:22:18.335084 17259 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f9cee260b5b45ab909619d7c28d8a2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:18.335179 17259 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f9cee260b5b45ab909619d7c28d8a2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:18.335292 17259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f9cee260b5b45ab909619d7c28d8a2f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:18.336380 17259 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f9cee260b5b45ab909619d7c28d8a2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f9cee260b5b45ab909619d7c28d8a2f" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 40657 } }
I20260808 02:22:18.336781 17259 leader_election.cc:302] T 00000000000000000000000000000000 P 1f9cee260b5b45ab909619d7c28d8a2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f9cee260b5b45ab909619d7c28d8a2f; no voters:
I20260808 02:22:18.337141 17259 leader_election.cc:288] T 00000000000000000000000000000000 P 1f9cee260b5b45ab909619d7c28d8a2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:22:18.337265 17265 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f9cee260b5b45ab909619d7c28d8a2f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:18.337518 17265 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f9cee260b5b45ab909619d7c28d8a2f [term 1 LEADER]: Becoming Leader. State: Replica: 1f9cee260b5b45ab909619d7c28d8a2f, State: Running, Role: LEADER
I20260808 02:22:18.337980 17265 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f9cee260b5b45ab909619d7c28d8a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f9cee260b5b45ab909619d7c28d8a2f" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 40657 } }
I20260808 02:22:18.338675 17259 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f9cee260b5b45ab909619d7c28d8a2f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:18.340010 17266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f9cee260b5b45ab909619d7c28d8a2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f9cee260b5b45ab909619d7c28d8a2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f9cee260b5b45ab909619d7c28d8a2f" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 40657 } } }
I20260808 02:22:18.340019 17267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f9cee260b5b45ab909619d7c28d8a2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f9cee260b5b45ab909619d7c28d8a2f. Latest consensus state: current_term: 1 leader_uuid: "1f9cee260b5b45ab909619d7c28d8a2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f9cee260b5b45ab909619d7c28d8a2f" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 40657 } } }
I20260808 02:22:18.340183 17267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f9cee260b5b45ab909619d7c28d8a2f [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:18.340183 17266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f9cee260b5b45ab909619d7c28d8a2f [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:18.340835 17276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:22:18.344441 17276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:22:18.351333 17276 catalog_manager.cc:1347] Generated new cluster ID: 3bd93ce1543343aa90623080531798b1
I20260808 02:22:18.351516 17276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:22:18.362303 17276 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:22:18.363579 17276 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:22:18.371071 17276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f9cee260b5b45ab909619d7c28d8a2f: Generated new TSK 0
I20260808 02:22:18.372064 17276 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.79.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
I20260808 02:22:18.411015 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:40657
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:38783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:18.542528 17293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:18.542896 17293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:18.543007 17293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:18.547663 17293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:18.547783 17293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:18.547876 17293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:22:18.554550 17293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:38783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:40657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:18.555979 17293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:18.557202 17293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:18.568899 17305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:18.568914 17303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:18.568949 17293 server_base.cc:1034] running on GCE node
W20260808 02:22:18.568899 17302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:18.569705 17293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:18.570472 17293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:18.571746 17293 hybrid_clock.cc:648] HybridClock initialized: now 1786155738571719 us; error 33 us; skew 500 ppm
Aug 08 02:22:18 dist-test-slave-1vmg krb5kdc[17149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155738, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:18.575084 17293 init.cc:377] Logged in from keytab as kudu/127.24.79.193@KRBTEST.COM (short username kudu)
I20260808 02:22:18.576326 17293 webserver.cc:460] Webserver started at http://127.24.79.193:33185/ using document root <none> and password file <none>
I20260808 02:22:18.577010 17293 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:18.577103 17293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:18.577354 17293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:18.579864 17293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "bfdbad158f6a43549d17654f112d9603"
format_stamp: "Formatted at 2026-08-08 02:22:18 on dist-test-slave-1vmg"
server_key: "71b399f2fb082abb92493115f1dc6e6c"
server_key_iv: "705b98658902cc760604a12341272747"
server_key_version: "encryptionkey@0"
I20260808 02:22:18.580552 17293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "bfdbad158f6a43549d17654f112d9603"
format_stamp: "Formatted at 2026-08-08 02:22:18 on dist-test-slave-1vmg"
server_key: "71b399f2fb082abb92493115f1dc6e6c"
server_key_iv: "705b98658902cc760604a12341272747"
server_key_version: "encryptionkey@0"
I20260808 02:22:18.585383 17293 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:22:18.588665 17314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:18.590509 17293 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:22:18.590691 17293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "bfdbad158f6a43549d17654f112d9603"
format_stamp: "Formatted at 2026-08-08 02:22:18 on dist-test-slave-1vmg"
server_key: "71b399f2fb082abb92493115f1dc6e6c"
server_key_iv: "705b98658902cc760604a12341272747"
server_key_version: "encryptionkey@0"
I20260808 02:22:18.590840 17293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:18.616128 17293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:18.619824 17293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:18.620035 17293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:18.620683 17293 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:22:18.621683 17293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:22:18.621739 17293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:18.621785 17293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:22:18.621807 17293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:18.632207 17293 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:36181
I20260808 02:22:18.632278 17467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:36181 every 8 connection(s)
I20260808 02:22:18.633271 17293 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:22:18.641062 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 17293
I20260808 02:22:18.641185 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:22:18.641485 24895 external_mini_cluster.cc:1428] Setting key 5b99b3d8d1220091b8631b3fdbf64446
Aug 08 02:22:18 dist-test-slave-1vmg krb5kdc[17149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155738, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:22:18.649886 17263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:18.635583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:50989 (local address 127.24.79.254:40657)
0808 02:22:18.635720 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:18.635724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:18.636792 (+ 1068us) server_negotiation.cc:407] Connection header received
0808 02:22:18.638318 (+ 1526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:18.638322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:18.638394 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:18.638474 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:18.640048 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:18.640669 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:18.642064 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:18.642390 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:18.642483 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:18.645728 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:18.645731 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:18.645733 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:18.645764 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:18.647213 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:18.647892 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:18.647894 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:18.647896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:18.647970 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:18.648384 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:18.648387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:18.648388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:18.648599 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:18.648688 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:18.649522 (+ 834us) server_negotiation.cc:299] Negotiation successful
0808 02:22:18.649673 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 02:22:18.651496 17472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:18.635935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:40657 (local address 127.24.79.193:50989)
0808 02:22:18.636546 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:18.636599 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:18.638045 (+ 1446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:18.638716 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:18.638725 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:18.639261 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:18.639868 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:18.639885 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:18.640821 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:18.640824 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:18.641879 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:18.641895 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:18.642598 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:18.642602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:18.642830 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:18.643641 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:22:18.643661 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:18.645480 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:18.647353 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:18.647362 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:18.647383 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:18.647797 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:18.648050 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:18.648053 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:18.648055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:18.648277 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:18.648730 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:18.648739 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:18.649175 (+ 436us) client_negotiation.cc:769] Sending connection context
0808 02:22:18.649440 (+ 265us) client_negotiation.cc:240] Negotiation successful
0808 02:22:18.650178 (+ 738us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":166,"threads_started":1}
I20260808 02:22:18.653254 17470 heartbeater.cc:347] Connected to a master server at 127.24.79.254:40657
I20260808 02:22:18.653575 17470 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.24.79.194@KRBTEST.COM; defaulting to no policy
I20260808 02:22:18.654215 17470 heartbeater.cc:511] Master 127.24.79.254:40657 requested a full tablet report, sending...
I20260808 02:22:18.656240 17213 ts_manager.cc:194] Registered new tserver with Master: bfdbad158f6a43549d17654f112d9603 (127.24.79.193:36181)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.194@KRBTEST.COM" created.
I20260808 02:22:18.657814 17213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.193@KRBTEST.COM'} at 127.24.79.193:50989
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
I20260808 02:22:18.679296 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:40657
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:38783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:18.813349 17477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:18.813686 17477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:18.813745 17477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:18.818068 17477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:18.818150 17477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:18.818215 17477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:22:18.824599 17477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:38783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:40657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:18.825882 17477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:18.827031 17477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:18.838781 17486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:18.838800 17484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:18.838781 17488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:18.839000 17477 server_base.cc:1034] running on GCE node
I20260808 02:22:18.839808 17477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:18.840565 17477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:18.841816 17477 hybrid_clock.cc:648] HybridClock initialized: now 1786155738841794 us; error 43 us; skew 500 ppm
Aug 08 02:22:18 dist-test-slave-1vmg krb5kdc[17149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155738, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:18.845252 17477 init.cc:377] Logged in from keytab as kudu/127.24.79.194@KRBTEST.COM (short username kudu)
I20260808 02:22:18.846427 17477 webserver.cc:460] Webserver started at http://127.24.79.194:46883/ using document root <none> and password file <none>
I20260808 02:22:18.847155 17477 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:18.847242 17477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:18.847488 17477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:18.849921 17477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "800ba371701246dc978a89e05271518d"
format_stamp: "Formatted at 2026-08-08 02:22:18 on dist-test-slave-1vmg"
server_key: "b39ebe28638f287478ee0f1a7cd3ca61"
server_key_iv: "8b989edc3ecb64da933fc7fa7444d8a4"
server_key_version: "encryptionkey@0"
I20260808 02:22:18.850656 17477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "800ba371701246dc978a89e05271518d"
format_stamp: "Formatted at 2026-08-08 02:22:18 on dist-test-slave-1vmg"
server_key: "b39ebe28638f287478ee0f1a7cd3ca61"
server_key_iv: "8b989edc3ecb64da933fc7fa7444d8a4"
server_key_version: "encryptionkey@0"
I20260808 02:22:18.855558 17477 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260808 02:22:18.858857 17498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:18.860508 17477 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:22:18.860702 17477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "800ba371701246dc978a89e05271518d"
format_stamp: "Formatted at 2026-08-08 02:22:18 on dist-test-slave-1vmg"
server_key: "b39ebe28638f287478ee0f1a7cd3ca61"
server_key_iv: "8b989edc3ecb64da933fc7fa7444d8a4"
server_key_version: "encryptionkey@0"
I20260808 02:22:18.860858 17477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:18.901846 17477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:18.905851 17477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:18.906090 17477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:18.907174 17477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:22:18.908622 17477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:22:18.908695 17477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:18.908766 17477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:22:18.908798 17477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:18.919839 17477 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:45109
I20260808 02:22:18.919870 17667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:45109 every 8 connection(s)
I20260808 02:22:18.921072 17477 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:22:18.930037 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 17477
I20260808 02:22:18.930191 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:22:18.930521 24895 external_mini_cluster.cc:1428] Setting key 99b4940249a5025e52c4253056f9e04b
Aug 08 02:22:18 dist-test-slave-1vmg krb5kdc[17149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155738, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:22:18.939424 17263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:18.923723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:53681 (local address 127.24.79.254:40657)
0808 02:22:18.923919 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:18.923923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:18.925054 (+ 1131us) server_negotiation.cc:407] Connection header received
0808 02:22:18.926523 (+ 1469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:18.926527 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:18.926597 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:18.926678 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:18.928254 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:18.928942 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:18.930681 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:18.930996 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:18.931110 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:18.934347 (+ 3237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:18.934352 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:18.934355 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:18.934392 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:18.936172 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:18.937059 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:18.937063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:18.937064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:18.937168 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:18.937644 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:18.937647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:18.937649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:18.937893 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:18.937985 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:18.939062 (+ 1077us) server_negotiation.cc:299] Negotiation successful
0808 02:22:18.939234 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260808 02:22:18.940915 17670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:18.924169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:40657 (local address 127.24.79.194:53681)
0808 02:22:18.924844 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:18.924899 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:18.926235 (+ 1336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:18.926954 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:18.926964 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:18.927420 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:18.928050 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:18.928067 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:18.929124 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:18.929127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:18.930445 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:18.930464 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:18.931211 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:18.931216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:18.931432 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:18.932309 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:22:18.932335 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:18.934103 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:18.936437 (+ 2334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:18.936446 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:18.936470 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:18.936913 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:18.937285 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:18.937289 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:18.937291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:18.937526 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:18.938030 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:18.938040 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:18.938532 (+ 492us) client_negotiation.cc:769] Sending connection context
0808 02:22:18.938946 (+ 414us) client_negotiation.cc:240] Negotiation successful
0808 02:22:18.939435 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":157,"threads_started":1}
I20260808 02:22:18.942792 17668 heartbeater.cc:347] Connected to a master server at 127.24.79.254:40657
WARNING: no policy specified for kudu/127.24.79.195@KRBTEST.COM; defaulting to no policy
I20260808 02:22:18.943192 17668 heartbeater.cc:464] Registering TS with master...
I20260808 02:22:18.943949 17668 heartbeater.cc:511] Master 127.24.79.254:40657 requested a full tablet report, sending...
I20260808 02:22:18.945533 17213 ts_manager.cc:194] Registered new tserver with Master: 800ba371701246dc978a89e05271518d (127.24.79.194:45109)
I20260808 02:22:18.946208 17213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.194@KRBTEST.COM'} at 127.24.79.194:53681
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
I20260808 02:22:18.975701 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:40657
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:38783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:19.114214 17674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:19.114595 17674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:19.114665 17674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:19.119582 17674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:19.119706 17674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:19.119817 17674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:22:19.126174 17674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:38783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:40657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:19.127588 17674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:19.129207 17674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:19.141659 17682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:19.141659 17685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:19.141665 17683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:19.142239 17674 server_base.cc:1034] running on GCE node
I20260808 02:22:19.142781 17674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:19.143664 17674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:19.144897 17674 hybrid_clock.cc:648] HybridClock initialized: now 1786155739144876 us; error 40 us; skew 500 ppm
Aug 08 02:22:19 dist-test-slave-1vmg krb5kdc[17149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155739, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:19.148653 17674 init.cc:377] Logged in from keytab as kudu/127.24.79.195@KRBTEST.COM (short username kudu)
I20260808 02:22:19.149982 17674 webserver.cc:460] Webserver started at http://127.24.79.195:39837/ using document root <none> and password file <none>
I20260808 02:22:19.150624 17674 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:19.150700 17674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:19.150964 17674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:19.153470 17674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "b108b3182367455a8fae4e7fa767fffa"
format_stamp: "Formatted at 2026-08-08 02:22:19 on dist-test-slave-1vmg"
server_key: "e2be1592409ef25c287ecc9c20f4d2af"
server_key_iv: "dd3783807e6c9e43f40a692a6865ed17"
server_key_version: "encryptionkey@0"
I20260808 02:22:19.154146 17674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "b108b3182367455a8fae4e7fa767fffa"
format_stamp: "Formatted at 2026-08-08 02:22:19 on dist-test-slave-1vmg"
server_key: "e2be1592409ef25c287ecc9c20f4d2af"
server_key_iv: "dd3783807e6c9e43f40a692a6865ed17"
server_key_version: "encryptionkey@0"
I20260808 02:22:19.159044 17674 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 02:22:19.162708 17693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:19.165102 17674 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260808 02:22:19.165339 17674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "b108b3182367455a8fae4e7fa767fffa"
format_stamp: "Formatted at 2026-08-08 02:22:19 on dist-test-slave-1vmg"
server_key: "e2be1592409ef25c287ecc9c20f4d2af"
server_key_iv: "dd3783807e6c9e43f40a692a6865ed17"
server_key_version: "encryptionkey@0"
I20260808 02:22:19.165516 17674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:19.194751 17674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:19.199258 17674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:19.199474 17674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:19.200169 17674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:22:19.201313 17674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:22:19.201371 17674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:19.201418 17674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:22:19.201438 17674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:19.214387 17674 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:38473
I20260808 02:22:19.214447 17863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:38473 every 8 connection(s)
I20260808 02:22:19.215543 17674 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:22:19.217289 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 17674
I20260808 02:22:19.217451 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:22:19.217752 24895 external_mini_cluster.cc:1428] Setting key c8943fb86ab4d8760254e6b60adef885
Aug 08 02:22:19 dist-test-slave-1vmg krb5kdc[17149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155739, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:22:19.234133 17263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:19.219147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:53469 (local address 127.24.79.254:40657)
0808 02:22:19.219306 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:19.219311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:19.220067 (+ 756us) server_negotiation.cc:407] Connection header received
0808 02:22:19.221152 (+ 1085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:19.221155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:19.221220 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:19.221300 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:19.222857 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.223551 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.224962 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.225224 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.225330 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:19.229000 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:19.229004 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:19.229007 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:19.229055 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:19.230711 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:19.231611 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:19.231615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:19.231617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:19.231738 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:19.232325 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:19.232330 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:19.232332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:19.232849 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:19.232969 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:19.233813 (+ 844us) server_negotiation.cc:299] Negotiation successful
0808 02:22:19.233956 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 02:22:19.235067 17867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:19.218644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:40657 (local address 127.24.79.195:53469)
0808 02:22:19.219883 (+ 1239us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:19.219950 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:19.220899 (+ 949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:19.221560 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:19.221571 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:19.222013 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:19.222642 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:19.222659 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.223744 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.223747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:19.224776 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:19.224789 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.225439 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.225444 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:19.225642 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:19.226615 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:22:19.226638 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:19.228674 (+ 2036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:19.230901 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:19.230911 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:19.230971 (+ 60us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:19.231482 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:19.231878 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:19.231881 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:19.231883 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:19.232144 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:19.233031 (+ 887us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:19.233041 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:19.233487 (+ 446us) client_negotiation.cc:769] Sending connection context
0808 02:22:19.233690 (+ 203us) client_negotiation.cc:240] Negotiation successful
0808 02:22:19.234043 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":965,"thread_start_us":773,"threads_started":1}
I20260808 02:22:19.236672 17865 heartbeater.cc:347] Connected to a master server at 127.24.79.254:40657
I20260808 02:22:19.236974 17865 heartbeater.cc:464] Registering TS with master...
I20260808 02:22:19.237640 17865 heartbeater.cc:511] Master 127.24.79.254:40657 requested a full tablet report, sending...
I20260808 02:22:19.239195 17213 ts_manager.cc:194] Registered new tserver with Master: b108b3182367455a8fae4e7fa767fffa (127.24.79.195:38473)
I20260808 02:22:19.239967 17213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.195@KRBTEST.COM'} at 127.24.79.195:53469
I20260808 02:22:19.246186 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:22:19.258152 17263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:19.248620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60444 (local address 127.24.79.254:40657)
0808 02:22:19.248808 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:19.248812 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:19.249035 (+ 223us) server_negotiation.cc:407] Connection header received
0808 02:22:19.249118 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:19.249120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:19.249173 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:19.249298 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:19.250304 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.250876 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.251969 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.252233 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.252339 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:19.253991 (+ 1652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:19.253995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:19.253998 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:19.254034 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:19.255810 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:19.256436 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:19.256440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:19.256441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:19.256548 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:19.256964 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:19.256968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:19.256969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:19.257334 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:19.257467 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:19.257839 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 02:22:19.257971 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 02:22:19.262471 17213 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60444:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:22:19.265962 17213 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:22:19.283354 17884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:19.277317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60244 (local address 127.24.79.193:36181)
0808 02:22:19.277725 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:19.277733 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:19.277760 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:22:19.278042 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:19.278049 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:19.278312 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:19.278485 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:19.279121 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.280255 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.281896 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.282513 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.282635 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:19.282841 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:19.283036 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 02:22:19.283165 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":79,"threads_started":1}
I20260808 02:22:19.283346 17889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:19.278004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:38473 (local address 127.0.0.1:37654)
0808 02:22:19.278358 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:19.278374 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:19.278462 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:19.279223 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:19.279226 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:19.279242 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:19.279359 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:19.279366 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.280521 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.280524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:19.281635 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:19.281644 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.282869 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.282873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:19.282999 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:19.283030 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 02:22:19.283092 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 02:22:19.283174 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":21,"thread_start_us":190,"threads_started":1}
I20260808 02:22:19.283339 17881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:19.277110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:36181 (local address 127.0.0.1:60244)
0808 02:22:19.277560 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:19.277642 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:19.277805 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:19.278709 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:19.278717 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:19.278743 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:19.278902 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:19.278909 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.280396 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.280400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:19.281764 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:19.281775 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.282654 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.282658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:19.282787 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:19.282829 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 02:22:19.282983 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 02:22:19.283166 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":38,"thread_start_us":135,"threads_started":1}
I20260808 02:22:19.284017 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:19.278222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37654 (local address 127.24.79.195:38473)
0808 02:22:19.278709 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:19.278722 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:19.278745 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:22:19.278819 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:19.278823 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:19.279060 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:19.279227 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:19.279458 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.280381 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.281788 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.282699 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.283428 (+ 729us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:19.283608 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:19.283665 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 02:22:19.283771 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":162,"threads_started":1}
I20260808 02:22:19.285344 17893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:19.278450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:45109 (local address 127.0.0.1:42422)
0808 02:22:19.278790 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:19.278804 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:19.278903 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:19.279828 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:19.279831 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:19.279851 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:19.279956 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:19.279961 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.281606 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.281610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:19.282743 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:19.282755 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.284922 (+ 2167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.284927 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:19.285047 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:19.285077 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 02:22:19.285155 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 02:22:19.285232 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":89,"threads_started":1}
I20260808 02:22:19.286024 17894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:19.278866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42422 (local address 127.24.79.194:45109)
0808 02:22:19.279372 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:19.279383 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:19.279405 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:22:19.279479 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:19.279484 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:19.279671 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:19.279848 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:19.280285 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.281443 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.282889 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.284692 (+ 1803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.284814 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:19.285769 (+ 955us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:19.285820 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 02:22:19.285908 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":162,"threads_started":1}
I20260808 02:22:19.287505 17767 tablet_service.cc:1467] Processing CreateTablet for tablet fa9f7bcea6144eb7b7a706ea93be290d (DEFAULT_TABLE table=test-table [id=800150dff9f54f7aa85f18a600571344]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:22:19.287506 17379 tablet_service.cc:1467] Processing CreateTablet for tablet fa9f7bcea6144eb7b7a706ea93be290d (DEFAULT_TABLE table=test-table [id=800150dff9f54f7aa85f18a600571344]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:22:19.288890 17767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa9f7bcea6144eb7b7a706ea93be290d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:19.288890 17379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa9f7bcea6144eb7b7a706ea93be290d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:19.289088 17574 tablet_service.cc:1467] Processing CreateTablet for tablet fa9f7bcea6144eb7b7a706ea93be290d (DEFAULT_TABLE table=test-table [id=800150dff9f54f7aa85f18a600571344]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:22:19.290091 17574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa9f7bcea6144eb7b7a706ea93be290d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:19.297664 17895 tablet_bootstrap.cc:492] T fa9f7bcea6144eb7b7a706ea93be290d P 800ba371701246dc978a89e05271518d: Bootstrap starting.
I20260808 02:22:19.298563 17896 tablet_bootstrap.cc:492] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603: Bootstrap starting.
I20260808 02:22:19.299305 17899 tablet_bootstrap.cc:492] T fa9f7bcea6144eb7b7a706ea93be290d P b108b3182367455a8fae4e7fa767fffa: Bootstrap starting.
I20260808 02:22:19.300279 17895 tablet_bootstrap.cc:654] T fa9f7bcea6144eb7b7a706ea93be290d P 800ba371701246dc978a89e05271518d: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:19.300997 17896 tablet_bootstrap.cc:654] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:19.301460 17895 log.cc:824] T fa9f7bcea6144eb7b7a706ea93be290d P 800ba371701246dc978a89e05271518d: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:19.301632 17899 tablet_bootstrap.cc:654] T fa9f7bcea6144eb7b7a706ea93be290d P b108b3182367455a8fae4e7fa767fffa: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:19.301909 17896 log.cc:824] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:19.302975 17899 log.cc:824] T fa9f7bcea6144eb7b7a706ea93be290d P b108b3182367455a8fae4e7fa767fffa: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:19.304244 17895 tablet_bootstrap.cc:492] T fa9f7bcea6144eb7b7a706ea93be290d P 800ba371701246dc978a89e05271518d: No bootstrap required, opened a new log
I20260808 02:22:19.304278 17896 tablet_bootstrap.cc:492] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603: No bootstrap required, opened a new log
I20260808 02:22:19.304474 17895 ts_tablet_manager.cc:1397] T fa9f7bcea6144eb7b7a706ea93be290d P 800ba371701246dc978a89e05271518d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 02:22:19.304473 17896 ts_tablet_manager.cc:1397] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260808 02:22:19.305300 17899 tablet_bootstrap.cc:492] T fa9f7bcea6144eb7b7a706ea93be290d P b108b3182367455a8fae4e7fa767fffa: No bootstrap required, opened a new log
I20260808 02:22:19.305532 17899 ts_tablet_manager.cc:1397] T fa9f7bcea6144eb7b7a706ea93be290d P b108b3182367455a8fae4e7fa767fffa: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 02:22:19.307791 17895 raft_consensus.cc:348] T fa9f7bcea6144eb7b7a706ea93be290d P 800ba371701246dc978a89e05271518d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfdbad158f6a43549d17654f112d9603" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 36181 } } peers { permanent_uuid: "b108b3182367455a8fae4e7fa767fffa" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38473 } } peers { permanent_uuid: "800ba371701246dc978a89e05271518d" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 45109 } }
I20260808 02:22:19.308291 17895 raft_consensus.cc:374] T fa9f7bcea6144eb7b7a706ea93be290d P 800ba371701246dc978a89e05271518d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:19.308414 17895 raft_consensus.cc:729] T fa9f7bcea6144eb7b7a706ea93be290d P 800ba371701246dc978a89e05271518d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 800ba371701246dc978a89e05271518d, State: Initialized, Role: FOLLOWER
I20260808 02:22:19.309262 17895 consensus_queue.cc:260] T fa9f7bcea6144eb7b7a706ea93be290d P 800ba371701246dc978a89e05271518d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfdbad158f6a43549d17654f112d9603" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 36181 } } peers { permanent_uuid: "b108b3182367455a8fae4e7fa767fffa" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38473 } } peers { permanent_uuid: "800ba371701246dc978a89e05271518d" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 45109 } }
I20260808 02:22:19.309428 17896 raft_consensus.cc:348] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfdbad158f6a43549d17654f112d9603" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 36181 } } peers { permanent_uuid: "b108b3182367455a8fae4e7fa767fffa" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38473 } } peers { permanent_uuid: "800ba371701246dc978a89e05271518d" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 45109 } }
I20260808 02:22:19.309821 17896 raft_consensus.cc:374] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:19.309900 17896 raft_consensus.cc:729] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfdbad158f6a43549d17654f112d9603, State: Initialized, Role: FOLLOWER
I20260808 02:22:19.309921 17899 raft_consensus.cc:348] T fa9f7bcea6144eb7b7a706ea93be290d P b108b3182367455a8fae4e7fa767fffa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfdbad158f6a43549d17654f112d9603" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 36181 } } peers { permanent_uuid: "b108b3182367455a8fae4e7fa767fffa" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38473 } } peers { permanent_uuid: "800ba371701246dc978a89e05271518d" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 45109 } }
I20260808 02:22:19.310201 17899 raft_consensus.cc:374] T fa9f7bcea6144eb7b7a706ea93be290d P b108b3182367455a8fae4e7fa767fffa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:19.310279 17899 raft_consensus.cc:729] T fa9f7bcea6144eb7b7a706ea93be290d P b108b3182367455a8fae4e7fa767fffa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b108b3182367455a8fae4e7fa767fffa, State: Initialized, Role: FOLLOWER
I20260808 02:22:19.310489 17896 consensus_queue.cc:260] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfdbad158f6a43549d17654f112d9603" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 36181 } } peers { permanent_uuid: "b108b3182367455a8fae4e7fa767fffa" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38473 } } peers { permanent_uuid: "800ba371701246dc978a89e05271518d" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 45109 } }
I20260808 02:22:19.310591 17895 ts_tablet_manager.cc:1428] T fa9f7bcea6144eb7b7a706ea93be290d P 800ba371701246dc978a89e05271518d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 02:22:19.310753 17899 consensus_queue.cc:260] T fa9f7bcea6144eb7b7a706ea93be290d P b108b3182367455a8fae4e7fa767fffa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfdbad158f6a43549d17654f112d9603" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 36181 } } peers { permanent_uuid: "b108b3182367455a8fae4e7fa767fffa" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38473 } } peers { permanent_uuid: "800ba371701246dc978a89e05271518d" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 45109 } }
I20260808 02:22:19.311096 17668 heartbeater.cc:503] Master 127.24.79.254:40657 was elected leader, sending a full tablet report...
I20260808 02:22:19.311666 17896 ts_tablet_manager.cc:1428] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260808 02:22:19.311664 17899 ts_tablet_manager.cc:1428] T fa9f7bcea6144eb7b7a706ea93be290d P b108b3182367455a8fae4e7fa767fffa: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 02:22:19.312044 17865 heartbeater.cc:503] Master 127.24.79.254:40657 was elected leader, sending a full tablet report...
I20260808 02:22:19.312104 17470 heartbeater.cc:503] Master 127.24.79.254:40657 was elected leader, sending a full tablet report...
I20260808 02:22:19.359665 17904 raft_consensus.cc:482] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:22:19.359962 17904 raft_consensus.cc:504] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfdbad158f6a43549d17654f112d9603" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 36181 } } peers { permanent_uuid: "b108b3182367455a8fae4e7fa767fffa" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38473 } } peers { permanent_uuid: "800ba371701246dc978a89e05271518d" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 45109 } }
I20260808 02:22:19.361369 17904 leader_election.cc:288] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b108b3182367455a8fae4e7fa767fffa (127.24.79.195:38473), 800ba371701246dc978a89e05271518d (127.24.79.194:45109)
I20260808 02:22:19.367516 17894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:19.362420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:49711 (local address 127.24.79.194:45109)
0808 02:22:19.362556 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:19.362562 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:19.362602 (+ 40us) server_negotiation.cc:407] Connection header received
0808 02:22:19.362743 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:19.362745 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:19.362800 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:19.362869 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:19.363178 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.363987 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.366029 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.366725 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.366849 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:19.366937 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:19.367306 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 02:22:19.367410 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 02:22:19.367525 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:19.361823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:36239 (local address 127.24.79.195:38473)
0808 02:22:19.361971 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:19.361975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:19.362324 (+ 349us) server_negotiation.cc:407] Connection header received
0808 02:22:19.362649 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:19.362653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:19.362717 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:19.362811 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:19.363178 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.363987 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.366029 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.366700 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.366849 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:19.366910 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:19.367323 (+ 413us) server_negotiation.cc:299] Negotiation successful
0808 02:22:19.367420 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:22:19.367525 17908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:19.362216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:45109 (local address 127.24.79.193:49711)
0808 02:22:19.362528 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:19.362541 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:19.362641 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:19.362971 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:19.362973 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:19.362989 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:19.363074 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:19.363079 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.364146 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.364150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:19.365840 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:19.365857 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.366884 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.366889 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:19.367074 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:19.367130 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 02:22:19.367224 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 02:22:19.367321 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":173,"thread_start_us":107,"threads_started":1}
I20260808 02:22:19.367970 17906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:19.361715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:38473 (local address 127.24.79.193:36239)
0808 02:22:19.362202 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:19.362219 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:19.362422 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:19.362829 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:19.362832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:19.362850 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:19.363062 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:19.363070 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.364146 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.364150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:19.365863 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:19.365870 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.366884 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.366889 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:19.367183 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:19.367204 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 02:22:19.367759 (+ 555us) client_negotiation.cc:240] Negotiation successful
0808 02:22:19.367849 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":80,"thread_start_us":159,"threads_started":1}
I20260808 02:22:19.368487 17598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa9f7bcea6144eb7b7a706ea93be290d" candidate_uuid: "bfdbad158f6a43549d17654f112d9603" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "800ba371701246dc978a89e05271518d" is_pre_election: true
I20260808 02:22:19.368732 17799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa9f7bcea6144eb7b7a706ea93be290d" candidate_uuid: "bfdbad158f6a43549d17654f112d9603" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b108b3182367455a8fae4e7fa767fffa" is_pre_election: true
I20260808 02:22:19.368824 17598 raft_consensus.cc:2454] T fa9f7bcea6144eb7b7a706ea93be290d P 800ba371701246dc978a89e05271518d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfdbad158f6a43549d17654f112d9603 in term 0.
I20260808 02:22:19.369048 17799 raft_consensus.cc:2454] T fa9f7bcea6144eb7b7a706ea93be290d P b108b3182367455a8fae4e7fa767fffa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfdbad158f6a43549d17654f112d9603 in term 0.
I20260808 02:22:19.369401 17317 leader_election.cc:302] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 800ba371701246dc978a89e05271518d, bfdbad158f6a43549d17654f112d9603; no voters:
I20260808 02:22:19.369690 17904 raft_consensus.cc:2781] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:22:19.369797 17904 raft_consensus.cc:482] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:22:19.369843 17904 raft_consensus.cc:3037] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:19.371071 17904 raft_consensus.cc:504] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfdbad158f6a43549d17654f112d9603" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 36181 } } peers { permanent_uuid: "b108b3182367455a8fae4e7fa767fffa" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38473 } } peers { permanent_uuid: "800ba371701246dc978a89e05271518d" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 45109 } }
I20260808 02:22:19.371466 17904 leader_election.cc:288] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603 [CANDIDATE]: Term 1 election: Requested vote from peers b108b3182367455a8fae4e7fa767fffa (127.24.79.195:38473), 800ba371701246dc978a89e05271518d (127.24.79.194:45109)
I20260808 02:22:19.372155 17799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa9f7bcea6144eb7b7a706ea93be290d" candidate_uuid: "bfdbad158f6a43549d17654f112d9603" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b108b3182367455a8fae4e7fa767fffa"
I20260808 02:22:19.372149 17598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa9f7bcea6144eb7b7a706ea93be290d" candidate_uuid: "bfdbad158f6a43549d17654f112d9603" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "800ba371701246dc978a89e05271518d"
I20260808 02:22:19.372402 17799 raft_consensus.cc:3037] T fa9f7bcea6144eb7b7a706ea93be290d P b108b3182367455a8fae4e7fa767fffa [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:19.372402 17598 raft_consensus.cc:3037] T fa9f7bcea6144eb7b7a706ea93be290d P 800ba371701246dc978a89e05271518d [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:19.373890 17799 raft_consensus.cc:2454] T fa9f7bcea6144eb7b7a706ea93be290d P b108b3182367455a8fae4e7fa767fffa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfdbad158f6a43549d17654f112d9603 in term 1.
I20260808 02:22:19.373890 17598 raft_consensus.cc:2454] T fa9f7bcea6144eb7b7a706ea93be290d P 800ba371701246dc978a89e05271518d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfdbad158f6a43549d17654f112d9603 in term 1.
I20260808 02:22:19.374373 17317 leader_election.cc:302] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 800ba371701246dc978a89e05271518d, bfdbad158f6a43549d17654f112d9603; no voters:
I20260808 02:22:19.374635 17904 raft_consensus.cc:2781] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:19.374980 17904 raft_consensus.cc:686] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603 [term 1 LEADER]: Becoming Leader. State: Replica: bfdbad158f6a43549d17654f112d9603, State: Running, Role: LEADER
I20260808 02:22:19.375392 17904 consensus_queue.cc:237] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfdbad158f6a43549d17654f112d9603" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 36181 } } peers { permanent_uuid: "b108b3182367455a8fae4e7fa767fffa" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38473 } } peers { permanent_uuid: "800ba371701246dc978a89e05271518d" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 45109 } }
I20260808 02:22:19.379670 17211 catalog_manager.cc:5495] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603 reported cstate change: term changed from 0 to 1, leader changed from <none> to bfdbad158f6a43549d17654f112d9603 (127.24.79.193). New cstate: current_term: 1 leader_uuid: "bfdbad158f6a43549d17654f112d9603" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfdbad158f6a43549d17654f112d9603" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 36181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b108b3182367455a8fae4e7fa767fffa" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "800ba371701246dc978a89e05271518d" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 45109 } health_report { overall_health: UNKNOWN } } }
W20260808 02:22:19.384928 17471 tablet.cc:2369] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:22:19.416658 17884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:19.412500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60256 (local address 127.24.79.193:36181)
0808 02:22:19.412657 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:19.412661 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:19.412694 (+ 33us) server_negotiation.cc:407] Connection header received
0808 02:22:19.412788 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:19.412790 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:19.412855 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:19.412949 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:22:19.413338 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.413883 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.414876 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.415184 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.415278 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:19.415693 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:22:19.416221 (+ 528us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:22:19.416323 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:19.416432 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 02:22:19.416499 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 02:22:19.422717 17669 tablet.cc:2369] T fa9f7bcea6144eb7b7a706ea93be290d P 800ba371701246dc978a89e05271518d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:22:19.425498 17598 raft_consensus.cc:1261] T fa9f7bcea6144eb7b7a706ea93be290d P 800ba371701246dc978a89e05271518d [term 1 FOLLOWER]: Refusing update from remote peer bfdbad158f6a43549d17654f112d9603: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:22:19.425498 17799 raft_consensus.cc:1261] T fa9f7bcea6144eb7b7a706ea93be290d P b108b3182367455a8fae4e7fa767fffa [term 1 FOLLOWER]: Refusing update from remote peer bfdbad158f6a43549d17654f112d9603: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:22:19.426589 17909 consensus_queue.cc:1035] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603 [LEADER]: Connected to new peer: Peer: permanent_uuid: "800ba371701246dc978a89e05271518d" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 45109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:22:19.426888 17904 consensus_queue.cc:1035] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b108b3182367455a8fae4e7fa767fffa" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 38473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:22:19.436033 17913 mvcc.cc:205] Tried to move back new op lower bound from 7316093908671393792 to 7316093908485906432. Current Snapshot: MvccSnapshot[applied={T|T < 7316093908671393792}]
I20260808 02:22:19.439716 17915 mvcc.cc:205] Tried to move back new op lower bound from 7316093908671393792 to 7316093908485906432. Current Snapshot: MvccSnapshot[applied={T|T < 7316093908671393792}]
I20260808 02:22:19.448400 17207 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:22:19.448653 17207 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:22:19.451850 17207 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f9cee260b5b45ab909619d7c28d8a2f: Sending DeleteTablet for 3 replicas of tablet fa9f7bcea6144eb7b7a706ea93be290d
I20260808 02:22:19.452837 17379 tablet_service.cc:1514] Processing DeleteTablet for tablet fa9f7bcea6144eb7b7a706ea93be290d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:22:19 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:60244
I20260808 02:22:19.453171 17767 tablet_service.cc:1514] Processing DeleteTablet for tablet fa9f7bcea6144eb7b7a706ea93be290d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:22:19 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:37654
I20260808 02:22:19.453462 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 17293
I20260808 02:22:19.453689 17925 tablet_replica.cc:330] T fa9f7bcea6144eb7b7a706ea93be290d P bfdbad158f6a43549d17654f112d9603: stopping tablet replica
I20260808 02:22:19.453753 17574 tablet_service.cc:1514] Processing DeleteTablet for tablet fa9f7bcea6144eb7b7a706ea93be290d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:22:19 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:42422
I20260808 02:22:19.454231 17927 tablet_replica.cc:330] T fa9f7bcea6144eb7b7a706ea93be290d P 800ba371701246dc978a89e05271518d: stopping tablet replica
I20260808 02:22:19.454618 17927 raft_consensus.cc:2229] T fa9f7bcea6144eb7b7a706ea93be290d P 800ba371701246dc978a89e05271518d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:22:19.455063 17927 raft_consensus.cc:2258] T fa9f7bcea6144eb7b7a706ea93be290d P 800ba371701246dc978a89e05271518d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:22:19.457840 17927 ts_tablet_manager.cc:1905] T fa9f7bcea6144eb7b7a706ea93be290d P 800ba371701246dc978a89e05271518d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:22:19.458652 17926 tablet_replica.cc:330] T fa9f7bcea6144eb7b7a706ea93be290d P b108b3182367455a8fae4e7fa767fffa: stopping tablet replica
I20260808 02:22:19.459147 17926 raft_consensus.cc:2229] T fa9f7bcea6144eb7b7a706ea93be290d P b108b3182367455a8fae4e7fa767fffa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:22:19.459663 17926 raft_consensus.cc:2258] T fa9f7bcea6144eb7b7a706ea93be290d P b108b3182367455a8fae4e7fa767fffa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:22:19.461354 17927 ts_tablet_manager.cc:1918] T fa9f7bcea6144eb7b7a706ea93be290d P 800ba371701246dc978a89e05271518d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:22:19.461480 17927 log.cc:1196] T fa9f7bcea6144eb7b7a706ea93be290d P 800ba371701246dc978a89e05271518d: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/fa9f7bcea6144eb7b7a706ea93be290d
I20260808 02:22:19.461858 17927 ts_tablet_manager.cc:1939] T fa9f7bcea6144eb7b7a706ea93be290d P 800ba371701246dc978a89e05271518d: Deleting consensus metadata
I20260808 02:22:19.462037 17926 ts_tablet_manager.cc:1905] T fa9f7bcea6144eb7b7a706ea93be290d P b108b3182367455a8fae4e7fa767fffa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:22:19.463730 17191 catalog_manager.cc:4841] TS 800ba371701246dc978a89e05271518d (127.24.79.194:45109): tablet fa9f7bcea6144eb7b7a706ea93be290d (table test-table [id=800150dff9f54f7aa85f18a600571344]) successfully deleted
I20260808 02:22:19.464928 17926 ts_tablet_manager.cc:1918] T fa9f7bcea6144eb7b7a706ea93be290d P b108b3182367455a8fae4e7fa767fffa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:22:19.465054 17926 log.cc:1196] T fa9f7bcea6144eb7b7a706ea93be290d P b108b3182367455a8fae4e7fa767fffa: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/fa9f7bcea6144eb7b7a706ea93be290d
I20260808 02:22:19.465505 17926 ts_tablet_manager.cc:1939] T fa9f7bcea6144eb7b7a706ea93be290d P b108b3182367455a8fae4e7fa767fffa: Deleting consensus metadata
I20260808 02:22:19.467156 17191 catalog_manager.cc:4841] TS b108b3182367455a8fae4e7fa767fffa (127.24.79.195:38473): tablet fa9f7bcea6144eb7b7a706ea93be290d (table test-table [id=800150dff9f54f7aa85f18a600571344]) successfully deleted
W20260808 02:22:19.470448 17191 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.79.193:36181 (error 108)
I20260808 02:22:19.470520 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 17477
I20260808 02:22:19.476648 17893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:19.476013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:36181 (local address 127.0.0.1:60272)
0808 02:22:19.476412 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:19.476522 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.193:36181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283}
W20260808 02:22:19.476986 17191 catalog_manager.cc:4584] TS bfdbad158f6a43549d17654f112d9603 (127.24.79.193:36181): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fa9f7bcea6144eb7b7a706ea93be290d: Network error: Client connection negotiation failed: client connection to 127.24.79.193:36181: connect: Connection refused (error 111)
I20260808 02:22:19.488718 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 17674
I20260808 02:22:19.511822 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 17168
2026-08-08T02:22:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2787 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T02:22:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:19Z Disabled control of system clock
I20260808 02:22:19.563628 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:45847
--webserver_interface=127.24.79.254
--webserver_port=0
--builtin_ntp_servers=127.24.79.212:41043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:45847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:22:19.695758 17936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:19.696089 17936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:19.696127 17936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:19.700531 17936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:19.700634 17936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:19.700649 17936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:19.700665 17936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:19.700680 17936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:19.705847 17936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:41043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:45847
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:45847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:19.707118 17936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:19.708662 17936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:19.718346 17946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:19.718488 17936 server_base.cc:1034] running on GCE node
W20260808 02:22:19.718349 17943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:19.718346 17944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:19.719389 17936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:19.720815 17936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:19.722055 17936 hybrid_clock.cc:648] HybridClock initialized: now 1786155739721983 us; error 91 us; skew 500 ppm
I20260808 02:22:19.724488 17936 webserver.cc:460] Webserver started at http://127.24.79.254:42913/ using document root <none> and password file <none>
I20260808 02:22:19.725162 17936 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:19.725256 17936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:19.725502 17936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:19.728008 17936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "fe39085754984db09c8f36ea27e42020"
format_stamp: "Formatted at 2026-08-08 02:22:19 on dist-test-slave-1vmg"
server_key: "50e0589028d7f903d0cd9ad8b1ab9ae4"
server_key_iv: "238053dcc327bfa682f060d599196e34"
server_key_version: "encryptionkey@0"
I20260808 02:22:19.728667 17936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "fe39085754984db09c8f36ea27e42020"
format_stamp: "Formatted at 2026-08-08 02:22:19 on dist-test-slave-1vmg"
server_key: "50e0589028d7f903d0cd9ad8b1ab9ae4"
server_key_iv: "238053dcc327bfa682f060d599196e34"
server_key_version: "encryptionkey@0"
I20260808 02:22:19.733510 17936 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:22:19.736686 17955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:19.738443 17936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:22:19.738613 17936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "fe39085754984db09c8f36ea27e42020"
format_stamp: "Formatted at 2026-08-08 02:22:19 on dist-test-slave-1vmg"
server_key: "50e0589028d7f903d0cd9ad8b1ab9ae4"
server_key_iv: "238053dcc327bfa682f060d599196e34"
server_key_version: "encryptionkey@0"
I20260808 02:22:19.738757 17936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:19.757593 17936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:19.758468 17936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:19.758684 17936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:19.766897 17936 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:45847
I20260808 02:22:19.767012 18035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:45847 every 8 connection(s)
I20260808 02:22:19.768085 17936 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:19.772146 18037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:19.773128 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 17936
I20260808 02:22:19.773281 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:19.773643 24895 external_mini_cluster.cc:1428] Setting key 7aca72ba02fdd329fae7b0f29b81b0ce
I20260808 02:22:19.780833 18037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe39085754984db09c8f36ea27e42020: Bootstrap starting.
I20260808 02:22:19.784142 18037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe39085754984db09c8f36ea27e42020: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:19.785132 18037 log.cc:824] T 00000000000000000000000000000000 P fe39085754984db09c8f36ea27e42020: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:19.785761 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:19.775366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46596 (local address 127.24.79.254:45847)
0808 02:22:19.776313 (+ 947us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:19.776333 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:19.776379 (+ 46us) server_negotiation.cc:407] Connection header received
0808 02:22:19.777247 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:19.777277 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:19.777779 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:19.778350 (+ 571us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:19.778898 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.780059 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.781260 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:19.782022 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:19.782263 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:19.783373 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:19.783381 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:19.783403 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:19.783979 (+ 576us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:19.784068 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:19.784101 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:19.784219 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:19.784483 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 02:22:19.784826 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":213,"threads_started":1}
I20260808 02:22:19.787644 18037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe39085754984db09c8f36ea27e42020: No bootstrap required, opened a new log
I20260808 02:22:19.791939 18037 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe39085754984db09c8f36ea27e42020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe39085754984db09c8f36ea27e42020" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45847 } }
I20260808 02:22:19.792243 18037 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe39085754984db09c8f36ea27e42020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:19.792284 18037 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe39085754984db09c8f36ea27e42020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe39085754984db09c8f36ea27e42020, State: Initialized, Role: FOLLOWER
I20260808 02:22:19.792764 18037 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe39085754984db09c8f36ea27e42020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe39085754984db09c8f36ea27e42020" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45847 } }
I20260808 02:22:19.792910 18037 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe39085754984db09c8f36ea27e42020 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:19.792964 18037 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe39085754984db09c8f36ea27e42020 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:19.793069 18037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe39085754984db09c8f36ea27e42020 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:19.794468 18037 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe39085754984db09c8f36ea27e42020 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe39085754984db09c8f36ea27e42020" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45847 } }
I20260808 02:22:19.794893 18037 leader_election.cc:302] T 00000000000000000000000000000000 P fe39085754984db09c8f36ea27e42020 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe39085754984db09c8f36ea27e42020; no voters:
I20260808 02:22:19.795324 18037 leader_election.cc:288] T 00000000000000000000000000000000 P fe39085754984db09c8f36ea27e42020 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:22:19.795516 18048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe39085754984db09c8f36ea27e42020 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:19.795804 18048 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe39085754984db09c8f36ea27e42020 [term 1 LEADER]: Becoming Leader. State: Replica: fe39085754984db09c8f36ea27e42020, State: Running, Role: LEADER
I20260808 02:22:19.796165 18048 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe39085754984db09c8f36ea27e42020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe39085754984db09c8f36ea27e42020" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45847 } }
I20260808 02:22:19.796734 18037 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe39085754984db09c8f36ea27e42020 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:19.797987 18050 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe39085754984db09c8f36ea27e42020 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe39085754984db09c8f36ea27e42020" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe39085754984db09c8f36ea27e42020" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45847 } } }
I20260808 02:22:19.798038 18051 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe39085754984db09c8f36ea27e42020 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe39085754984db09c8f36ea27e42020. Latest consensus state: current_term: 1 leader_uuid: "fe39085754984db09c8f36ea27e42020" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe39085754984db09c8f36ea27e42020" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 45847 } } }
I20260808 02:22:19.798156 18050 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe39085754984db09c8f36ea27e42020 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:19.798156 18051 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe39085754984db09c8f36ea27e42020 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:19.798655 18059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:22:19.802443 18059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:22:19.810360 18059 catalog_manager.cc:1347] Generated new cluster ID: eb2ae8e7997745308e3b3d250c31591e
I20260808 02:22:19.810477 18059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:22:19.821368 18059 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:22:19.822594 18059 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:22:19.827646 18059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe39085754984db09c8f36ea27e42020: Generated new TSK 0
I20260808 02:22:19.828511 18059 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:22:19.839215 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:45847
--builtin_ntp_servers=127.24.79.212:41043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:22:19.975281 18071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:19.975622 18071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:19.975667 18071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:19.980489 18071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:19.980615 18071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:19.980680 18071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:22:19.986368 18071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:41043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:45847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:19.988125 18071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:19.989925 18071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:20.003751 18080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:20.003733 18079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:20.003713 18083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:20.004472 18071 server_base.cc:1034] running on GCE node
I20260808 02:22:20.005108 18071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:20.005955 18071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:20.009523 18071 hybrid_clock.cc:648] HybridClock initialized: now 1786155740009383 us; error 79 us; skew 500 ppm
I20260808 02:22:20.012755 18071 webserver.cc:460] Webserver started at http://127.24.79.193:45483/ using document root <none> and password file <none>
I20260808 02:22:20.013550 18071 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:20.013654 18071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:20.014110 18071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:20.016978 18071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "f9e58d1d07b042bfabc36e773237c400"
format_stamp: "Formatted at 2026-08-08 02:22:20 on dist-test-slave-1vmg"
server_key: "6e7dbf6c0034b2eac4db2122982141f7"
server_key_iv: "ca321fa26ec7209fcad2454a69988908"
server_key_version: "encryptionkey@0"
I20260808 02:22:20.017761 18071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "f9e58d1d07b042bfabc36e773237c400"
format_stamp: "Formatted at 2026-08-08 02:22:20 on dist-test-slave-1vmg"
server_key: "6e7dbf6c0034b2eac4db2122982141f7"
server_key_iv: "ca321fa26ec7209fcad2454a69988908"
server_key_version: "encryptionkey@0"
I20260808 02:22:20.023054 18071 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 02:22:20.026535 18090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:20.028723 18071 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:22:20.028928 18071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "f9e58d1d07b042bfabc36e773237c400"
format_stamp: "Formatted at 2026-08-08 02:22:20 on dist-test-slave-1vmg"
server_key: "6e7dbf6c0034b2eac4db2122982141f7"
server_key_iv: "ca321fa26ec7209fcad2454a69988908"
server_key_version: "encryptionkey@0"
I20260808 02:22:20.029230 18071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:20.040532 18071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:20.041409 18071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:20.041644 18071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:20.042362 18071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:22:20.043807 18071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:22:20.043918 18071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:20.043992 18071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:22:20.044135 18071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:20.054143 18071 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:44191
I20260808 02:22:20.054360 18248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:44191 every 8 connection(s)
I20260808 02:22:20.055279 18071 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:22:20.059540 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 18071
I20260808 02:22:20.059674 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:22:20.060001 24895 external_mini_cluster.cc:1428] Setting key 445795462a1e98c0eef10b08b20b6bdd
I20260808 02:22:20.062415 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:45847
--builtin_ntp_servers=127.24.79.212:41043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 02:22:20.068449 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:20.057687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:43711 (local address 127.24.79.254:45847)
0808 02:22:20.057858 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:20.057861 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:20.059542 (+ 1681us) server_negotiation.cc:407] Connection header received
0808 02:22:20.061462 (+ 1920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:20.061466 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:20.061525 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:20.061660 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:20.062831 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.063622 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.065547 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.065795 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.065915 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:20.067678 (+ 1763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:20.067682 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:20.067684 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:20.067703 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:20.067735 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:20.067752 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:20.067829 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:20.068192 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 02:22:20.068284 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 02:22:20.070015 18256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:20.058118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45847 (local address 127.24.79.193:43711)
0808 02:22:20.059341 (+ 1223us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:20.059396 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:20.061185 (+ 1789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:20.061897 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:20.061906 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:20.062014 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:20.062655 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:20.062680 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.063812 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.063816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:20.065298 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:20.065318 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.066010 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.066015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:20.066251 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:20.067186 (+ 935us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:22:20.067189 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:20.067266 (+ 77us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:22:20.067382 (+ 116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:20.067871 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:20.067889 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:20.067899 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 02:22:20.068096 (+ 197us) client_negotiation.cc:240] Negotiation successful
0808 02:22:20.068522 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":148,"threads_started":1}
I20260808 02:22:20.072186 18251 heartbeater.cc:347] Connected to a master server at 127.24.79.254:45847
I20260808 02:22:20.072561 18251 heartbeater.cc:464] Registering TS with master...
I20260808 02:22:20.073439 18251 heartbeater.cc:511] Master 127.24.79.254:45847 requested a full tablet report, sending...
I20260808 02:22:20.075806 17976 ts_manager.cc:194] Registered new tserver with Master: f9e58d1d07b042bfabc36e773237c400 (127.24.79.193:44191)
I20260808 02:22:20.077142 17976 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.79.193:43711
W20260808 02:22:20.200362 18257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:20.200707 18257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:20.200769 18257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:20.205233 18257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:20.205338 18257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:20.205422 18257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:22:20.210768 18257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:41043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:45847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:20.212147 18257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:20.213755 18257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:20.225869 18266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:20.225904 18263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:20.225976 18264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:20.226519 18257 server_base.cc:1034] running on GCE node
I20260808 02:22:20.227161 18257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:20.228019 18257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:20.229277 18257 hybrid_clock.cc:648] HybridClock initialized: now 1786155740229240 us; error 56 us; skew 500 ppm
I20260808 02:22:20.231799 18257 webserver.cc:460] Webserver started at http://127.24.79.194:33521/ using document root <none> and password file <none>
I20260808 02:22:20.232527 18257 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:20.232631 18257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:20.232860 18257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:20.235373 18257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "eb9572f8cf63410fb7bb62071dc6223e"
format_stamp: "Formatted at 2026-08-08 02:22:20 on dist-test-slave-1vmg"
server_key: "e5c038cf2a18ae9eca346dc2c0c1b7de"
server_key_iv: "4fc76d3d8fc97e465a4e0772eb4a15c8"
server_key_version: "encryptionkey@0"
I20260808 02:22:20.236009 18257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "eb9572f8cf63410fb7bb62071dc6223e"
format_stamp: "Formatted at 2026-08-08 02:22:20 on dist-test-slave-1vmg"
server_key: "e5c038cf2a18ae9eca346dc2c0c1b7de"
server_key_iv: "4fc76d3d8fc97e465a4e0772eb4a15c8"
server_key_version: "encryptionkey@0"
I20260808 02:22:20.240330 18257 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260808 02:22:20.243436 18278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:20.245407 18257 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:22:20.245661 18257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "eb9572f8cf63410fb7bb62071dc6223e"
format_stamp: "Formatted at 2026-08-08 02:22:20 on dist-test-slave-1vmg"
server_key: "e5c038cf2a18ae9eca346dc2c0c1b7de"
server_key_iv: "4fc76d3d8fc97e465a4e0772eb4a15c8"
server_key_version: "encryptionkey@0"
I20260808 02:22:20.245829 18257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:20.266477 18257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:20.267805 18257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:20.268070 18257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:20.269122 18257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:22:20.270964 18257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:22:20.271054 18257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260808 02:22:20.271121 18257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:22:20.271153 18257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:20.285596 18257 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:42939
I20260808 02:22:20.285602 18451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:42939 every 8 connection(s)
I20260808 02:22:20.287169 18257 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:22:20.295535 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 18257
I20260808 02:22:20.295675 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:22:20.296053 24895 external_mini_cluster.cc:1428] Setting key cfea12e5003284b4e01e47e8eaeb9df4
I20260808 02:22:20.298880 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:45847
--builtin_ntp_servers=127.24.79.212:41043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 02:22:20.300386 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:20.289862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:46797 (local address 127.24.79.254:45847)
0808 02:22:20.290006 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:20.290010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:20.291681 (+ 1671us) server_negotiation.cc:407] Connection header received
0808 02:22:20.293308 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:20.293313 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:20.293388 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:20.293505 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:20.294583 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.295561 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.297505 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.297789 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.297893 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:20.299751 (+ 1858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:20.299756 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:20.299777 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:20.299795 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:20.299822 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:20.299833 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:20.299881 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:20.300136 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 02:22:20.300226 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 02:22:20.302131 18456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:20.290527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45847 (local address 127.24.79.194:46797)
0808 02:22:20.291487 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:20.291534 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:20.293000 (+ 1466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:20.293672 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:20.293680 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:20.293776 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:20.294398 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:20.294415 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.295770 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.295774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:20.297363 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:20.297378 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.298090 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.298095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:20.298416 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:20.299342 (+ 926us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:22:20.299345 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:20.299414 (+ 69us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:22:20.299526 (+ 112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:20.299955 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:20.299966 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:20.299972 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 02:22:20.300793 (+ 821us) client_negotiation.cc:240] Negotiation successful
0808 02:22:20.301232 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":260,"threads_started":1}
I20260808 02:22:20.303902 18453 heartbeater.cc:347] Connected to a master server at 127.24.79.254:45847
I20260808 02:22:20.304207 18453 heartbeater.cc:464] Registering TS with master...
I20260808 02:22:20.304872 18453 heartbeater.cc:511] Master 127.24.79.254:45847 requested a full tablet report, sending...
I20260808 02:22:20.306300 17976 ts_manager.cc:194] Registered new tserver with Master: eb9572f8cf63410fb7bb62071dc6223e (127.24.79.194:42939)
I20260808 02:22:20.307183 17976 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.79.194:46797
W20260808 02:22:20.428995 18458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:20.429337 18458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:20.429378 18458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:20.433897 18458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:20.433982 18458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:20.434043 18458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:22:20.439363 18458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:41043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:45847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:20.440696 18458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:20.442229 18458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:20.453930 18465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:20.453929 18468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:20.454030 18458 server_base.cc:1034] running on GCE node
W20260808 02:22:20.453929 18466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:20.454944 18458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:20.455737 18458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:20.456931 18458 hybrid_clock.cc:648] HybridClock initialized: now 1786155740456881 us; error 70 us; skew 500 ppm
I20260808 02:22:20.459517 18458 webserver.cc:460] Webserver started at http://127.24.79.195:46573/ using document root <none> and password file <none>
I20260808 02:22:20.460273 18458 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:20.460357 18458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:20.460597 18458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:20.463006 18458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "a9323529b49b4119babffc1270d9d24e"
format_stamp: "Formatted at 2026-08-08 02:22:20 on dist-test-slave-1vmg"
server_key: "d9844508678c97f3084b1df50276756d"
server_key_iv: "0a21ee7d3ed4b66524703d06be67dde8"
server_key_version: "encryptionkey@0"
I20260808 02:22:20.463672 18458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "a9323529b49b4119babffc1270d9d24e"
format_stamp: "Formatted at 2026-08-08 02:22:20 on dist-test-slave-1vmg"
server_key: "d9844508678c97f3084b1df50276756d"
server_key_iv: "0a21ee7d3ed4b66524703d06be67dde8"
server_key_version: "encryptionkey@0"
I20260808 02:22:20.468264 18458 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:22:20.471432 18482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:20.473062 18458 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:22:20.473225 18458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "a9323529b49b4119babffc1270d9d24e"
format_stamp: "Formatted at 2026-08-08 02:22:20 on dist-test-slave-1vmg"
server_key: "d9844508678c97f3084b1df50276756d"
server_key_iv: "0a21ee7d3ed4b66524703d06be67dde8"
server_key_version: "encryptionkey@0"
I20260808 02:22:20.473385 18458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:20.496603 18458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:20.497592 18458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:20.497855 18458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:20.498631 18458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:22:20.499905 18458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:22:20.499972 18458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:20.500020 18458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:22:20.500048 18458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:20.509505 18458 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:37673
I20260808 02:22:20.509568 18640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:37673 every 8 connection(s)
I20260808 02:22:20.510550 18458 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:22:20.519662 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 18458
I20260808 02:22:20.519834 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:22:20.520151 24895 external_mini_cluster.cc:1428] Setting key f3ae6f224da6bdd9226137df285c5f47
I20260808 02:22:20.522614 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:20.512927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:36421 (local address 127.24.79.254:45847)
0808 02:22:20.513072 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:20.513075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:20.514495 (+ 1420us) server_negotiation.cc:407] Connection header received
0808 02:22:20.516236 (+ 1741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:20.516241 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:20.516306 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:20.516613 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:20.517660 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.518537 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.520263 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.520518 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.520623 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:20.522010 (+ 1387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:20.522013 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:20.522016 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:20.522029 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:20.522054 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:20.522065 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:20.522133 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:20.522380 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 02:22:20.522452 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:22:20.523783 18647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:20.513360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:45847 (local address 127.24.79.195:36421)
0808 02:22:20.514303 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:20.514356 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:20.515909 (+ 1553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:20.516748 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:20.516757 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:20.516866 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:20.517458 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:20.517474 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.518745 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.518748 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:20.520085 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:20.520102 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.520721 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.520726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:20.520935 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:20.521665 (+ 730us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:22:20.521667 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:20.521733 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:22:20.521826 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:20.522157 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:20.522169 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:20.522176 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 02:22:20.522330 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 02:22:20.522711 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":170,"threads_started":1}
I20260808 02:22:20.525358 18642 heartbeater.cc:347] Connected to a master server at 127.24.79.254:45847
I20260808 02:22:20.525702 18642 heartbeater.cc:464] Registering TS with master...
I20260808 02:22:20.526474 18642 heartbeater.cc:511] Master 127.24.79.254:45847 requested a full tablet report, sending...
I20260808 02:22:20.528103 17976 ts_manager.cc:194] Registered new tserver with Master: a9323529b49b4119babffc1270d9d24e (127.24.79.195:37673)
I20260808 02:22:20.528949 17976 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.79.195:36421
I20260808 02:22:20.535682 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:22:20.542802 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:20.537951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46604 (local address 127.24.79.254:45847)
0808 02:22:20.538120 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:20.538124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:20.538258 (+ 134us) server_negotiation.cc:407] Connection header received
0808 02:22:20.538424 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:20.538427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:20.538479 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:20.538568 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:20.539025 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.539663 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.540754 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.540989 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.541103 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:20.542164 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:20.542167 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:20.542170 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:20.542188 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:20.542214 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:20.542229 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:20.542305 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:20.542552 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 02:22:20.542640 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 02:22:20.547010 17976 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46604:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:22:20.550875 17976 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:22:20.568397 18661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:20.561833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:37673 (local address 127.0.0.1:37412)
0808 02:22:20.562350 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:20.562408 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:20.562552 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:20.563811 (+ 1259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:20.563819 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:20.563843 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:20.564011 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:20.564019 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.565485 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.565489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:20.566499 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:20.566512 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.567745 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.567749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:20.567882 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:20.567923 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 02:22:20.568044 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 02:22:20.568204 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":59,"thread_start_us":96,"threads_started":1}
I20260808 02:22:20.568418 18662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:20.562025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:42939 (local address 127.0.0.1:48790)
0808 02:22:20.562979 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:20.562994 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:20.563093 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:20.564075 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:20.564078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:20.564098 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:20.564211 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:20.564218 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.565337 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.565341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:20.566862 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:20.566874 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.568079 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.568083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:20.568180 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:20.568203 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:22:20.568264 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 02:22:20.568325 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":850,"mutex_wait_us":46,"thread_start_us":198,"threads_started":1}
I20260808 02:22:20.568665 18666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:20.562699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:44191 (local address 127.0.0.1:44268)
0808 02:22:20.563287 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:20.563299 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:20.563389 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:20.564256 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:20.564259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:20.564276 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:20.564372 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:20.564380 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.565935 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.565938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:20.567090 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:20.567099 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.568267 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.568271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:20.568373 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:20.568398 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 02:22:20.568448 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 02:22:20.568520 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":107,"threads_started":1}
I20260808 02:22:20.568955 18664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:20.562026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37412 (local address 127.24.79.195:37673)
0808 02:22:20.563256 (+ 1230us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:20.563271 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:20.563298 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:22:20.563384 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:20.563391 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:20.563640 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:20.563824 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:20.564157 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.565355 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.566655 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.567610 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.567717 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:20.567954 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:20.568654 (+ 700us) server_negotiation.cc:299] Negotiation successful
0808 02:22:20.568780 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1096,"thread_start_us":211,"threads_started":1}
I20260808 02:22:20.569344 18669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:20.563178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44268 (local address 127.24.79.193:44191)
0808 02:22:20.563611 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:20.563618 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:20.563643 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:22:20.563724 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:20.563729 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:20.563915 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:20.564082 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:20.564517 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.565620 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.567227 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.568123 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.568797 (+ 674us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:20.569001 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:20.569075 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 02:22:20.569171 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":77,"threads_started":1}
I20260808 02:22:20.569343 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:20.562681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48790 (local address 127.24.79.194:42939)
0808 02:22:20.563612 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:20.563623 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:20.563643 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:22:20.563724 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:20.563729 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:20.563915 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:20.564082 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:20.564330 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.565193 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.567046 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.567895 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.568059 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:20.568293 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:20.568354 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 02:22:20.568472 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":828,"thread_start_us":157,"threads_started":1}
I20260808 02:22:20.572395 18548 tablet_service.cc:1467] Processing CreateTablet for tablet e3a3d0c14d9f46c99dd66c7dbcd80f90 (DEFAULT_TABLE table=test-table [id=701d0e3a3d9f4832aaf1e8d248e437f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:22:20.572453 18362 tablet_service.cc:1467] Processing CreateTablet for tablet e3a3d0c14d9f46c99dd66c7dbcd80f90 (DEFAULT_TABLE table=test-table [id=701d0e3a3d9f4832aaf1e8d248e437f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:22:20.572450 18166 tablet_service.cc:1467] Processing CreateTablet for tablet e3a3d0c14d9f46c99dd66c7dbcd80f90 (DEFAULT_TABLE table=test-table [id=701d0e3a3d9f4832aaf1e8d248e437f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:22:20.573556 18548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3a3d0c14d9f46c99dd66c7dbcd80f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:20.573580 18166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3a3d0c14d9f46c99dd66c7dbcd80f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:20.573571 18362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3a3d0c14d9f46c99dd66c7dbcd80f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:20.581305 18672 tablet_bootstrap.cc:492] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P a9323529b49b4119babffc1270d9d24e: Bootstrap starting.
I20260808 02:22:20.583258 18674 tablet_bootstrap.cc:492] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400: Bootstrap starting.
I20260808 02:22:20.584409 18672 tablet_bootstrap.cc:654] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P a9323529b49b4119babffc1270d9d24e: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:20.585423 18672 log.cc:824] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P a9323529b49b4119babffc1270d9d24e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:20.585748 18674 tablet_bootstrap.cc:654] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:20.585779 18675 tablet_bootstrap.cc:492] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P eb9572f8cf63410fb7bb62071dc6223e: Bootstrap starting.
I20260808 02:22:20.586683 18674 log.cc:824] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:20.588208 18675 tablet_bootstrap.cc:654] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P eb9572f8cf63410fb7bb62071dc6223e: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:20.588217 18672 tablet_bootstrap.cc:492] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P a9323529b49b4119babffc1270d9d24e: No bootstrap required, opened a new log
I20260808 02:22:20.588452 18672 ts_tablet_manager.cc:1397] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P a9323529b49b4119babffc1270d9d24e: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 02:22:20.588997 18675 log.cc:824] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P eb9572f8cf63410fb7bb62071dc6223e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:20.589067 18674 tablet_bootstrap.cc:492] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400: No bootstrap required, opened a new log
I20260808 02:22:20.589265 18674 ts_tablet_manager.cc:1397] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 02:22:20.591207 18675 tablet_bootstrap.cc:492] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P eb9572f8cf63410fb7bb62071dc6223e: No bootstrap required, opened a new log
I20260808 02:22:20.591437 18675 ts_tablet_manager.cc:1397] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P eb9572f8cf63410fb7bb62071dc6223e: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.001s
I20260808 02:22:20.593196 18672 raft_consensus.cc:348] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P a9323529b49b4119babffc1270d9d24e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9323529b49b4119babffc1270d9d24e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37673 } } peers { permanent_uuid: "eb9572f8cf63410fb7bb62071dc6223e" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 42939 } } peers { permanent_uuid: "f9e58d1d07b042bfabc36e773237c400" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 44191 } }
I20260808 02:22:20.593537 18672 raft_consensus.cc:374] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P a9323529b49b4119babffc1270d9d24e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:20.593596 18672 raft_consensus.cc:729] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P a9323529b49b4119babffc1270d9d24e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9323529b49b4119babffc1270d9d24e, State: Initialized, Role: FOLLOWER
I20260808 02:22:20.593513 18674 raft_consensus.cc:348] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9323529b49b4119babffc1270d9d24e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37673 } } peers { permanent_uuid: "eb9572f8cf63410fb7bb62071dc6223e" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 42939 } } peers { permanent_uuid: "f9e58d1d07b042bfabc36e773237c400" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 44191 } }
I20260808 02:22:20.593799 18674 raft_consensus.cc:374] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:20.593937 18674 raft_consensus.cc:729] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e58d1d07b042bfabc36e773237c400, State: Initialized, Role: FOLLOWER
I20260808 02:22:20.594146 18672 consensus_queue.cc:260] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P a9323529b49b4119babffc1270d9d24e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9323529b49b4119babffc1270d9d24e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37673 } } peers { permanent_uuid: "eb9572f8cf63410fb7bb62071dc6223e" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 42939 } } peers { permanent_uuid: "f9e58d1d07b042bfabc36e773237c400" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 44191 } }
I20260808 02:22:20.594355 18674 consensus_queue.cc:260] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9323529b49b4119babffc1270d9d24e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37673 } } peers { permanent_uuid: "eb9572f8cf63410fb7bb62071dc6223e" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 42939 } } peers { permanent_uuid: "f9e58d1d07b042bfabc36e773237c400" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 44191 } }
I20260808 02:22:20.595186 18251 heartbeater.cc:503] Master 127.24.79.254:45847 was elected leader, sending a full tablet report...
I20260808 02:22:20.595395 18672 ts_tablet_manager.cc:1428] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P a9323529b49b4119babffc1270d9d24e: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260808 02:22:20.595395 18674 ts_tablet_manager.cc:1428] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 02:22:20.595368 18675 raft_consensus.cc:348] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P eb9572f8cf63410fb7bb62071dc6223e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9323529b49b4119babffc1270d9d24e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37673 } } peers { permanent_uuid: "eb9572f8cf63410fb7bb62071dc6223e" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 42939 } } peers { permanent_uuid: "f9e58d1d07b042bfabc36e773237c400" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 44191 } }
I20260808 02:22:20.595644 18675 raft_consensus.cc:374] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P eb9572f8cf63410fb7bb62071dc6223e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:20.595707 18675 raft_consensus.cc:729] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P eb9572f8cf63410fb7bb62071dc6223e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb9572f8cf63410fb7bb62071dc6223e, State: Initialized, Role: FOLLOWER
I20260808 02:22:20.595759 18642 heartbeater.cc:503] Master 127.24.79.254:45847 was elected leader, sending a full tablet report...
I20260808 02:22:20.596163 18675 consensus_queue.cc:260] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P eb9572f8cf63410fb7bb62071dc6223e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9323529b49b4119babffc1270d9d24e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37673 } } peers { permanent_uuid: "eb9572f8cf63410fb7bb62071dc6223e" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 42939 } } peers { permanent_uuid: "f9e58d1d07b042bfabc36e773237c400" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 44191 } }
I20260808 02:22:20.597168 18453 heartbeater.cc:503] Master 127.24.79.254:45847 was elected leader, sending a full tablet report...
I20260808 02:22:20.597391 18675 ts_tablet_manager.cc:1428] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P eb9572f8cf63410fb7bb62071dc6223e: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260808 02:22:20.641075 18684 raft_consensus.cc:482] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:22:20.641336 18684 raft_consensus.cc:504] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9323529b49b4119babffc1270d9d24e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37673 } } peers { permanent_uuid: "eb9572f8cf63410fb7bb62071dc6223e" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 42939 } } peers { permanent_uuid: "f9e58d1d07b042bfabc36e773237c400" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 44191 } }
I20260808 02:22:20.643271 18684 leader_election.cc:288] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9323529b49b4119babffc1270d9d24e (127.24.79.195:37673), eb9572f8cf63410fb7bb62071dc6223e (127.24.79.194:42939)
I20260808 02:22:20.647194 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:20.643091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:53941 (local address 127.24.79.194:42939)
0808 02:22:20.643282 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:20.643288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:20.643714 (+ 426us) server_negotiation.cc:407] Connection header received
0808 02:22:20.643927 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:20.643930 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:20.643978 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:20.644044 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:20.644382 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.644910 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.646158 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.646592 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.646713 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:20.646774 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:20.646963 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 02:22:20.647049 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260808 02:22:20.647214 18688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:20.642870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:42939 (local address 127.24.79.193:53941)
0808 02:22:20.643607 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:20.643620 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:20.643803 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:20.644072 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:20.644075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:20.644091 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:20.644259 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:20.644265 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.645050 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.645054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:20.646005 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:20.646018 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.646713 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.646717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:20.646811 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:20.646835 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 02:22:20.646905 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 02:22:20.647038 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"mutex_wait_us":56,"spinlock_wait_cycles":15616,"thread_start_us":100,"threads_started":1}
I20260808 02:22:20.647337 18664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:20.643186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:40829 (local address 127.24.79.195:37673)
0808 02:22:20.643309 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:20.643314 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:20.643688 (+ 374us) server_negotiation.cc:407] Connection header received
0808 02:22:20.643940 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:20.643942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:20.643991 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:20.644066 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:20.644420 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.644984 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.646334 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.646737 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.646851 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:20.646904 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:20.647123 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 02:22:20.647205 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:22:20.647599 18687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:20.642870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:37673 (local address 127.24.79.193:40829)
0808 02:22:20.643538 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:20.643558 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:20.643790 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:20.644094 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:20.644097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:20.644115 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:20.644295 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:20.644302 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.645116 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.645120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:20.646179 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:20.646191 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.646873 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.646876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:20.646998 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:20.647023 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 02:22:20.647091 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 02:22:20.647446 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":23,"thread_start_us":174,"threads_started":1}
I20260808 02:22:20.648100 18386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3a3d0c14d9f46c99dd66c7dbcd80f90" candidate_uuid: "f9e58d1d07b042bfabc36e773237c400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb9572f8cf63410fb7bb62071dc6223e" is_pre_election: true
I20260808 02:22:20.648295 18576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3a3d0c14d9f46c99dd66c7dbcd80f90" candidate_uuid: "f9e58d1d07b042bfabc36e773237c400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9323529b49b4119babffc1270d9d24e" is_pre_election: true
I20260808 02:22:20.648487 18386 raft_consensus.cc:2454] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P eb9572f8cf63410fb7bb62071dc6223e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9e58d1d07b042bfabc36e773237c400 in term 0.
I20260808 02:22:20.648619 18576 raft_consensus.cc:2454] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P a9323529b49b4119babffc1270d9d24e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9e58d1d07b042bfabc36e773237c400 in term 0.
I20260808 02:22:20.649258 18093 leader_election.cc:302] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eb9572f8cf63410fb7bb62071dc6223e, f9e58d1d07b042bfabc36e773237c400; no voters:
I20260808 02:22:20.649595 18684 raft_consensus.cc:2781] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:22:20.649691 18684 raft_consensus.cc:482] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:22:20.649752 18684 raft_consensus.cc:3037] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:20.650949 18684 raft_consensus.cc:504] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9323529b49b4119babffc1270d9d24e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37673 } } peers { permanent_uuid: "eb9572f8cf63410fb7bb62071dc6223e" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 42939 } } peers { permanent_uuid: "f9e58d1d07b042bfabc36e773237c400" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 44191 } }
I20260808 02:22:20.651418 18684 leader_election.cc:288] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400 [CANDIDATE]: Term 1 election: Requested vote from peers a9323529b49b4119babffc1270d9d24e (127.24.79.195:37673), eb9572f8cf63410fb7bb62071dc6223e (127.24.79.194:42939)
I20260808 02:22:20.651932 18576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3a3d0c14d9f46c99dd66c7dbcd80f90" candidate_uuid: "f9e58d1d07b042bfabc36e773237c400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9323529b49b4119babffc1270d9d24e"
I20260808 02:22:20.651952 18386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3a3d0c14d9f46c99dd66c7dbcd80f90" candidate_uuid: "f9e58d1d07b042bfabc36e773237c400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb9572f8cf63410fb7bb62071dc6223e"
I20260808 02:22:20.652117 18386 raft_consensus.cc:3037] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P eb9572f8cf63410fb7bb62071dc6223e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:20.652125 18576 raft_consensus.cc:3037] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P a9323529b49b4119babffc1270d9d24e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:20.653618 18386 raft_consensus.cc:2454] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P eb9572f8cf63410fb7bb62071dc6223e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e58d1d07b042bfabc36e773237c400 in term 1.
I20260808 02:22:20.653618 18576 raft_consensus.cc:2454] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P a9323529b49b4119babffc1270d9d24e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e58d1d07b042bfabc36e773237c400 in term 1.
I20260808 02:22:20.654104 18094 leader_election.cc:302] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a9323529b49b4119babffc1270d9d24e, eb9572f8cf63410fb7bb62071dc6223e, f9e58d1d07b042bfabc36e773237c400; no voters:
I20260808 02:22:20.654325 18684 raft_consensus.cc:2781] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:20.654587 18684 raft_consensus.cc:686] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400 [term 1 LEADER]: Becoming Leader. State: Replica: f9e58d1d07b042bfabc36e773237c400, State: Running, Role: LEADER
I20260808 02:22:20.655016 18684 consensus_queue.cc:237] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9323529b49b4119babffc1270d9d24e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37673 } } peers { permanent_uuid: "eb9572f8cf63410fb7bb62071dc6223e" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 42939 } } peers { permanent_uuid: "f9e58d1d07b042bfabc36e773237c400" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 44191 } }
I20260808 02:22:20.658970 17973 catalog_manager.cc:5495] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9e58d1d07b042bfabc36e773237c400 (127.24.79.193). New cstate: current_term: 1 leader_uuid: "f9e58d1d07b042bfabc36e773237c400" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9323529b49b4119babffc1270d9d24e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb9572f8cf63410fb7bb62071dc6223e" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 42939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e58d1d07b042bfabc36e773237c400" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 44191 } health_report { overall_health: HEALTHY } } }
I20260808 02:22:20.679986 18669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:20.675456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44270 (local address 127.24.79.193:44191)
0808 02:22:20.675590 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:20.675595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:20.675615 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:22:20.675710 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:20.675712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:20.675775 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:20.675876 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:22:20.676282 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.676805 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.677906 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:20.678273 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:20.678377 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:20.678809 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:22:20.679470 (+ 661us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:22:20.679589 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:20.679782 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 02:22:20.679854 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 02:22:20.687747 18386 raft_consensus.cc:1261] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P eb9572f8cf63410fb7bb62071dc6223e [term 1 FOLLOWER]: Refusing update from remote peer f9e58d1d07b042bfabc36e773237c400: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:22:20.687742 18576 raft_consensus.cc:1261] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P a9323529b49b4119babffc1270d9d24e [term 1 FOLLOWER]: Refusing update from remote peer f9e58d1d07b042bfabc36e773237c400: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:22:20.688862 18684 consensus_queue.cc:1035] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb9572f8cf63410fb7bb62071dc6223e" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 42939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:22:20.689175 18689 consensus_queue.cc:1035] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P f9e58d1d07b042bfabc36e773237c400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9323529b49b4119babffc1270d9d24e" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:22:20.697964 18692 mvcc.cc:205] Tried to move back new op lower bound from 7316093913846075392 to 7316093913726214144. Current Snapshot: MvccSnapshot[applied={T|T < 7316093913846075392}]
I20260808 02:22:20.700946 18694 mvcc.cc:205] Tried to move back new op lower bound from 7316093913846075392 to 7316093913726214144. Current Snapshot: MvccSnapshot[applied={T|T < 7316093913846075392}]
I20260808 02:22:20.710338 17976 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:22:20.710570 17976 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:22:20.714085 17976 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fe39085754984db09c8f36ea27e42020: Sending DeleteTablet for 3 replicas of tablet e3a3d0c14d9f46c99dd66c7dbcd80f90
I20260808 02:22:20.715157 18548 tablet_service.cc:1514] Processing DeleteTablet for tablet e3a3d0c14d9f46c99dd66c7dbcd80f90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:22:20 UTC) from {username='slave'} at 127.0.0.1:37412
I20260808 02:22:20.715232 18362 tablet_service.cc:1514] Processing DeleteTablet for tablet e3a3d0c14d9f46c99dd66c7dbcd80f90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:22:20 UTC) from {username='slave'} at 127.0.0.1:48790
I20260808 02:22:20.715801 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 18071
I20260808 02:22:20.718385 18709 tablet_replica.cc:330] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P eb9572f8cf63410fb7bb62071dc6223e: stopping tablet replica
I20260808 02:22:20.718385 18707 tablet_replica.cc:330] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P a9323529b49b4119babffc1270d9d24e: stopping tablet replica
I20260808 02:22:20.718847 18707 raft_consensus.cc:2229] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P a9323529b49b4119babffc1270d9d24e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:22:20.718848 18709 raft_consensus.cc:2229] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P eb9572f8cf63410fb7bb62071dc6223e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:22:20.719307 18707 raft_consensus.cc:2258] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P a9323529b49b4119babffc1270d9d24e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:22:20.719307 18709 raft_consensus.cc:2258] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P eb9572f8cf63410fb7bb62071dc6223e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:22:20.720996 18709 ts_tablet_manager.cc:1905] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P eb9572f8cf63410fb7bb62071dc6223e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:22:20.724120 18709 ts_tablet_manager.cc:1918] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P eb9572f8cf63410fb7bb62071dc6223e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:22:20.724272 18709 log.cc:1196] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P eb9572f8cf63410fb7bb62071dc6223e: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/e3a3d0c14d9f46c99dd66c7dbcd80f90
I20260808 02:22:20.724635 18709 ts_tablet_manager.cc:1939] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P eb9572f8cf63410fb7bb62071dc6223e: Deleting consensus metadata
I20260808 02:22:20.725286 18707 ts_tablet_manager.cc:1905] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P a9323529b49b4119babffc1270d9d24e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:22:20.726274 17961 catalog_manager.cc:4841] TS eb9572f8cf63410fb7bb62071dc6223e (127.24.79.194:42939): tablet e3a3d0c14d9f46c99dd66c7dbcd80f90 (table test-table [id=701d0e3a3d9f4832aaf1e8d248e437f3]) successfully deleted
I20260808 02:22:20.728417 18707 ts_tablet_manager.cc:1918] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P a9323529b49b4119babffc1270d9d24e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:22:20.728543 18707 log.cc:1196] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P a9323529b49b4119babffc1270d9d24e: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/e3a3d0c14d9f46c99dd66c7dbcd80f90
I20260808 02:22:20.728824 18707 ts_tablet_manager.cc:1939] T e3a3d0c14d9f46c99dd66c7dbcd80f90 P a9323529b49b4119babffc1270d9d24e: Deleting consensus metadata
I20260808 02:22:20.730538 17962 catalog_manager.cc:4841] TS a9323529b49b4119babffc1270d9d24e (127.24.79.195:37673): tablet e3a3d0c14d9f46c99dd66c7dbcd80f90 (table test-table [id=701d0e3a3d9f4832aaf1e8d248e437f3]) successfully deleted
I20260808 02:22:20.732538 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 18257
W20260808 02:22:20.732782 17961 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.79.193:44191 (error 108)
I20260808 02:22:20.740756 18666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:20.740370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:44191 (local address 127.0.0.1:44286)
0808 02:22:20.740568 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:20.740659 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.193:44191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260808 02:22:20.740974 17961 catalog_manager.cc:4584] TS f9e58d1d07b042bfabc36e773237c400 (127.24.79.193:44191): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e3a3d0c14d9f46c99dd66c7dbcd80f90: Network error: Client connection negotiation failed: client connection to 127.24.79.193:44191: connect: Connection refused (error 111)
I20260808 02:22:20.746260 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 18458
I20260808 02:22:20.764256 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 17936
2026-08-08T02:22:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1243 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:22:20 dist-test-slave-1vmg krb5kdc[18716](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:22:20 dist-test-slave-1vmg krb5kdc[18716](info): set up 2 sockets
Aug 08 02:22:20 dist-test-slave-1vmg krb5kdc[18716](info): commencing operation
krb5kdc: starting...
W20260808 02:22:21.948465 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:22:22 dist-test-slave-1vmg krb5kdc[18716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:22:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:22:22.108722 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:34555
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:34555
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:22.244334 18735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:22.244678 18735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:22.244722 18735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:22.249444 18735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:22.249567 18735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:22.249588 18735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:22.249616 18735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:22.249639 18735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:22.257153 18735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:34555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:34555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:22.258569 18735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:22.259877 18735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:22.269842 18746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:22.269855 18742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:22.269994 18735 server_base.cc:1034] running on GCE node
W20260808 02:22:22.269855 18743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:22.271076 18735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:22.272454 18735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:22.273693 18735 hybrid_clock.cc:648] HybridClock initialized: now 1786155742273666 us; error 49 us; skew 500 ppm
Aug 08 02:22:22 dist-test-slave-1vmg krb5kdc[18716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155742, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:22.280874 18735 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:22:22.282230 18735 webserver.cc:460] Webserver started at http://127.24.79.254:35617/ using document root <none> and password file <none>
I20260808 02:22:22.282955 18735 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:22.283056 18735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:22.283324 18735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:22.285727 18735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "117e5d13d0b14ae09ed4d89f0c0a13ff"
format_stamp: "Formatted at 2026-08-08 02:22:22 on dist-test-slave-1vmg"
server_key: "cd15fd38a0ca1375294a09d2ad84d1d4"
server_key_iv: "6f58b030f2cbd174b551bc1dc7469b04"
server_key_version: "encryptionkey@0"
I20260808 02:22:22.286417 18735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "117e5d13d0b14ae09ed4d89f0c0a13ff"
format_stamp: "Formatted at 2026-08-08 02:22:22 on dist-test-slave-1vmg"
server_key: "cd15fd38a0ca1375294a09d2ad84d1d4"
server_key_iv: "6f58b030f2cbd174b551bc1dc7469b04"
server_key_version: "encryptionkey@0"
I20260808 02:22:22.291127 18735 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260808 02:22:22.294353 18758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:22.296023 18735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:22:22.296195 18735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "117e5d13d0b14ae09ed4d89f0c0a13ff"
format_stamp: "Formatted at 2026-08-08 02:22:22 on dist-test-slave-1vmg"
server_key: "cd15fd38a0ca1375294a09d2ad84d1d4"
server_key_iv: "6f58b030f2cbd174b551bc1dc7469b04"
server_key_version: "encryptionkey@0"
I20260808 02:22:22.296334 18735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:22.309755 18735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:22.313726 18735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:22.313968 18735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:22.322775 18735 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:34555
I20260808 02:22:22.322831 18841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:34555 every 8 connection(s)
I20260808 02:22:22.323966 18735 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:22.327467 18842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:22.328393 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 18735
I20260808 02:22:22.328647 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:22.329075 24895 external_mini_cluster.cc:1428] Setting key e73fd7128ae0395f036023f887aefbfe
I20260808 02:22:22.336905 18842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff: Bootstrap starting.
I20260808 02:22:22.340130 18842 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff: Neither blocks nor log segments found. Creating new log.
Aug 08 02:22:22 dist-test-slave-1vmg krb5kdc[18716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:22:22.341023 18842 log.cc:824] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:22.343518 18842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff: No bootstrap required, opened a new log
I20260808 02:22:22.346216 18846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:22.331196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48724 (local address 127.24.79.254:34555)
0808 02:22:22.331725 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:22.331743 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:22.331786 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:22:22.332496 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:22.332517 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:22.332916 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:22.333308 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:22.334632 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:22.335860 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:22.337140 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:22.337725 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:22.337873 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:22.341047 (+ 3174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:22.341053 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:22.341074 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:22.341117 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:22.342998 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:22.343518 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:22.343524 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:22.343530 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:22.343708 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:22.344009 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:22.344013 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:22.344014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:22.344481 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:22.344692 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:22.345017 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 02:22:22.345363 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":160,"threads_started":1}
I20260808 02:22:22.347664 18842 raft_consensus.cc:348] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "117e5d13d0b14ae09ed4d89f0c0a13ff" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 34555 } }
I20260808 02:22:22.348012 18842 raft_consensus.cc:374] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:22.348090 18842 raft_consensus.cc:729] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117e5d13d0b14ae09ed4d89f0c0a13ff, State: Initialized, Role: FOLLOWER
I20260808 02:22:22.348635 18842 consensus_queue.cc:260] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "117e5d13d0b14ae09ed4d89f0c0a13ff" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 34555 } }
I20260808 02:22:22.348826 18842 raft_consensus.cc:388] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:22.348946 18842 raft_consensus.cc:482] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:22.349061 18842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:22.350199 18842 raft_consensus.cc:504] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "117e5d13d0b14ae09ed4d89f0c0a13ff" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 34555 } }
I20260808 02:22:22.350572 18842 leader_election.cc:302] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 117e5d13d0b14ae09ed4d89f0c0a13ff; no voters:
I20260808 02:22:22.350955 18842 leader_election.cc:288] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:22:22.351106 18848 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:22.351688 18848 raft_consensus.cc:686] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff [term 1 LEADER]: Becoming Leader. State: Replica: 117e5d13d0b14ae09ed4d89f0c0a13ff, State: Running, Role: LEADER
I20260808 02:22:22.352137 18848 consensus_queue.cc:237] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "117e5d13d0b14ae09ed4d89f0c0a13ff" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 34555 } }
I20260808 02:22:22.352468 18842 sys_catalog.cc:564] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:22.354023 18850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 117e5d13d0b14ae09ed4d89f0c0a13ff. Latest consensus state: current_term: 1 leader_uuid: "117e5d13d0b14ae09ed4d89f0c0a13ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "117e5d13d0b14ae09ed4d89f0c0a13ff" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 34555 } } }
I20260808 02:22:22.354205 18850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:22.354031 18849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "117e5d13d0b14ae09ed4d89f0c0a13ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "117e5d13d0b14ae09ed4d89f0c0a13ff" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 34555 } } }
I20260808 02:22:22.354446 18849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff [sys.catalog]: This master's current role is: LEADER
W20260808 02:22:22.357784 18873 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:22:22.357898 18873 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:22:22.357992 18869 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:22:22.359076 18869 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:22:22.366125 18869 catalog_manager.cc:1347] Generated new cluster ID: 7eb96ad3890544888552c029f153f884
I20260808 02:22:22.366235 18869 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:22:22.388265 18869 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:22:22.389407 18869 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:22:22.396754 18869 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff: Generated new TSK 0
I20260808 02:22:22.397652 18869 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.79.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
I20260808 02:22:22.435105 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:34555
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:22.565583 18881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:22.565905 18881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:22.565950 18881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:22.570479 18881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:22.570616 18881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:22.570713 18881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:22:22.576694 18881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:34555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:22.577984 18881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:22.579173 18881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:22.590754 18896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:22.590731 18891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:22.590731 18892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:22.590867 18881 server_base.cc:1034] running on GCE node
I20260808 02:22:22.591656 18881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:22.592432 18881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:22.593683 18881 hybrid_clock.cc:648] HybridClock initialized: now 1786155742593661 us; error 38 us; skew 500 ppm
Aug 08 02:22:22 dist-test-slave-1vmg krb5kdc[18716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155742, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:22.597170 18881 init.cc:377] Logged in from keytab as kudu/127.24.79.193@KRBTEST.COM (short username kudu)
I20260808 02:22:22.598470 18881 webserver.cc:460] Webserver started at http://127.24.79.193:40817/ using document root <none> and password file <none>
I20260808 02:22:22.599303 18881 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:22.599490 18881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:22.599835 18881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:22.602595 18881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "e271b6d6febe4d26b315ab8d16296274"
format_stamp: "Formatted at 2026-08-08 02:22:22 on dist-test-slave-1vmg"
server_key: "88230551d4c2cd1e3ad9fcaf974343bc"
server_key_iv: "f5fed608c0c2f6d7ceaa66e8ab26d5f8"
server_key_version: "encryptionkey@0"
I20260808 02:22:22.603328 18881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "e271b6d6febe4d26b315ab8d16296274"
format_stamp: "Formatted at 2026-08-08 02:22:22 on dist-test-slave-1vmg"
server_key: "88230551d4c2cd1e3ad9fcaf974343bc"
server_key_iv: "f5fed608c0c2f6d7ceaa66e8ab26d5f8"
server_key_version: "encryptionkey@0"
I20260808 02:22:22.608012 18881 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:22:22.611220 18903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:22.612843 18881 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:22:22.613013 18881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "e271b6d6febe4d26b315ab8d16296274"
format_stamp: "Formatted at 2026-08-08 02:22:22 on dist-test-slave-1vmg"
server_key: "88230551d4c2cd1e3ad9fcaf974343bc"
server_key_iv: "f5fed608c0c2f6d7ceaa66e8ab26d5f8"
server_key_version: "encryptionkey@0"
I20260808 02:22:22.613158 18881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:22.630789 18881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:22.635603 18881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:22.635851 18881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:22.636647 18881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:22:22.637735 18881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:22:22.637795 18881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:22.637856 18881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:22:22.637885 18881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:22.647946 18881 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:37189
I20260808 02:22:22.647979 19062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:37189 every 8 connection(s)
I20260808 02:22:22.649046 18881 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:22:22.655357 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 18881
I20260808 02:22:22.655519 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:22:22.655834 24895 external_mini_cluster.cc:1428] Setting key a2092f7bfee8e73410f3d685bd696996
Aug 08 02:22:22 dist-test-slave-1vmg krb5kdc[18716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155742, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:22:22.667390 18846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:22.651352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:54873 (local address 127.24.79.254:34555)
0808 02:22:22.651550 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:22.651555 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:22.652562 (+ 1007us) server_negotiation.cc:407] Connection header received
0808 02:22:22.654063 (+ 1501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:22.654066 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:22.654136 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:22.654369 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:22.656186 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:22.656831 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:22.658429 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:22.658748 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:22.658869 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:22.662407 (+ 3538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:22.662412 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:22.662415 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:22.662453 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:22.664411 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:22.665223 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:22.665228 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:22.665229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:22.665359 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:22.665847 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:22.665851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:22.665852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:22.666135 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:22.666264 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:22.667025 (+ 761us) server_negotiation.cc:299] Negotiation successful
0808 02:22:22.667195 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
WARNING: no policy specified for kudu/127.24.79.194@KRBTEST.COM; defaulting to no policy
I20260808 02:22:22.668447 19067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:22.651779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:34555 (local address 127.24.79.193:54873)
0808 02:22:22.652368 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:22.652430 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:22.653789 (+ 1359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:22.654740 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:22.654750 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:22.655341 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:22.655995 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:22.656014 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:22.657004 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:22.657008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:22.658257 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:22.658271 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:22.659012 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:22.659018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:22.659223 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:22.659918 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:22:22.659939 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:22.662094 (+ 2155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:22.664670 (+ 2576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:22.664679 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:22.664701 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:22.665074 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:22.665494 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:22.665497 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:22.665499 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:22.665735 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:22.666303 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:22.666311 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:22.666672 (+ 361us) client_negotiation.cc:769] Sending connection context
0808 02:22:22.666871 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 02:22:22.667399 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":173,"threads_started":1}
I20260808 02:22:22.670498 19063 heartbeater.cc:347] Connected to a master server at 127.24.79.254:34555
I20260808 02:22:22.670857 19063 heartbeater.cc:464] Registering TS with master...
I20260808 02:22:22.671540 19063 heartbeater.cc:511] Master 127.24.79.254:34555 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.194@KRBTEST.COM" created.
I20260808 02:22:22.673655 18786 ts_manager.cc:194] Registered new tserver with Master: e271b6d6febe4d26b315ab8d16296274 (127.24.79.193:37189)
I20260808 02:22:22.674897 18786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.193@KRBTEST.COM'} at 127.24.79.193:54873
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
I20260808 02:22:22.697539 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:34555
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:22.827883 19074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:22.828212 19074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:22.828259 19074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:22.832535 19074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:22.832635 19074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:22.832706 19074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:22:22.838703 19074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:34555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:22.840088 19074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:22.841311 19074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:22.852591 19082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:22.852742 19074 server_base.cc:1034] running on GCE node
W20260808 02:22:22.852604 19087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:22.852591 19083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:22.853502 19074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:22.854370 19074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:22.855611 19074 hybrid_clock.cc:648] HybridClock initialized: now 1786155742855553 us; error 46 us; skew 500 ppm
Aug 08 02:22:22 dist-test-slave-1vmg krb5kdc[18716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155742, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:22.859375 19074 init.cc:377] Logged in from keytab as kudu/127.24.79.194@KRBTEST.COM (short username kudu)
I20260808 02:22:22.860553 19074 webserver.cc:460] Webserver started at http://127.24.79.194:42117/ using document root <none> and password file <none>
I20260808 02:22:22.861196 19074 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:22.861269 19074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:22.861469 19074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:22.863971 19074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "c12b99125a4f4a998260914679cf4039"
format_stamp: "Formatted at 2026-08-08 02:22:22 on dist-test-slave-1vmg"
server_key: "1e6da0b30338d9d65eae1b3deb471dcf"
server_key_iv: "ecfb8b1df3dbd125532fc13670761f60"
server_key_version: "encryptionkey@0"
I20260808 02:22:22.864655 19074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "c12b99125a4f4a998260914679cf4039"
format_stamp: "Formatted at 2026-08-08 02:22:22 on dist-test-slave-1vmg"
server_key: "1e6da0b30338d9d65eae1b3deb471dcf"
server_key_iv: "ecfb8b1df3dbd125532fc13670761f60"
server_key_version: "encryptionkey@0"
I20260808 02:22:22.869362 19074 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:22:22.872900 19097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:22.874672 19074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:22:22.874869 19074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "c12b99125a4f4a998260914679cf4039"
format_stamp: "Formatted at 2026-08-08 02:22:22 on dist-test-slave-1vmg"
server_key: "1e6da0b30338d9d65eae1b3deb471dcf"
server_key_iv: "ecfb8b1df3dbd125532fc13670761f60"
server_key_version: "encryptionkey@0"
I20260808 02:22:22.875087 19074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:22.887339 19074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:22.891602 19074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:22.891844 19074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:22.892760 19074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:22:22.893826 19074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:22:22.893888 19074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:22.893949 19074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:22:22.893980 19074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:22.904295 19074 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:42927
I20260808 02:22:22.904373 19257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:42927 every 8 connection(s)
I20260808 02:22:22.905411 19074 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:22:22.907027 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 19074
I20260808 02:22:22.907217 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:22:22.907539 24895 external_mini_cluster.cc:1428] Setting key 34478a992912f3fc74843117c16d37e5
Aug 08 02:22:22 dist-test-slave-1vmg krb5kdc[18716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155742, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.79.195@KRBTEST.COM; defaulting to no policy
I20260808 02:22:22.923027 18846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:22.907768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:38797 (local address 127.24.79.254:34555)
0808 02:22:22.907967 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:22.907971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:22.909218 (+ 1247us) server_negotiation.cc:407] Connection header received
0808 02:22:22.910991 (+ 1773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:22.910995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:22.911060 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:22.911146 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:22.912627 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:22.913184 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:22.914530 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:22.914772 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:22.914874 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:22.918408 (+ 3534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:22.918411 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:22.918413 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:22.918447 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:22.920267 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:22.920932 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:22.920936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:22.920937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:22.921042 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:22.921480 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:22.921483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:22.921484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:22.921720 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:22.921820 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:22.922497 (+ 677us) server_negotiation.cc:299] Negotiation successful
0808 02:22:22.922800 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.195@KRBTEST.COM" created.
I20260808 02:22:22.924077 19260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:22.908345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:34555 (local address 127.24.79.194:38797)
0808 02:22:22.909021 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:22.909087 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:22.910668 (+ 1581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:22.911384 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:22.911393 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:22.911850 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:22.912434 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:22.912450 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:22.913352 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:22.913355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:22.914372 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:22.914386 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:22.915004 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:22.915011 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:22.915203 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:22.915992 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:22:22.916013 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:22.918143 (+ 2130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:22.920440 (+ 2297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:22.920447 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:22.920468 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:22.920802 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:22.921152 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:22.921155 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:22.921157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:22.921366 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:22.921845 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:22.921855 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:22.922206 (+ 351us) client_negotiation.cc:769] Sending connection context
0808 02:22:22.922418 (+ 212us) client_negotiation.cc:240] Negotiation successful
0808 02:22:22.923024 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":185,"threads_started":1}
I20260808 02:22:22.925590 19258 heartbeater.cc:347] Connected to a master server at 127.24.79.254:34555
I20260808 02:22:22.925915 19258 heartbeater.cc:464] Registering TS with master...
I20260808 02:22:22.926656 19258 heartbeater.cc:511] Master 127.24.79.254:34555 requested a full tablet report, sending...
I20260808 02:22:22.928051 18786 ts_manager.cc:194] Registered new tserver with Master: c12b99125a4f4a998260914679cf4039 (127.24.79.194:42927)
I20260808 02:22:22.928676 18786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.194@KRBTEST.COM'} at 127.24.79.194:38797
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
I20260808 02:22:22.950619 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:34555
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:37803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:23.089001 19263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:23.089342 19263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:23.089389 19263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:23.093747 19263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:23.093849 19263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:23.093912 19263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:22:23.099892 19263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:37803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:34555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:23.101292 19263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:23.102487 19263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:22:23.114974 19263 server_base.cc:1034] running on GCE node
W20260808 02:22:23.114897 19271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:23.114873 19273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:23.114872 19277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:23.115816 19263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:23.116828 19263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:23.120194 19263 hybrid_clock.cc:648] HybridClock initialized: now 1786155743120137 us; error 52 us; skew 500 ppm
Aug 08 02:22:23 dist-test-slave-1vmg krb5kdc[18716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155743, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:23.124125 19263 init.cc:377] Logged in from keytab as kudu/127.24.79.195@KRBTEST.COM (short username kudu)
I20260808 02:22:23.125349 19263 webserver.cc:460] Webserver started at http://127.24.79.195:35859/ using document root <none> and password file <none>
I20260808 02:22:23.126041 19263 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:23.126183 19263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:23.126452 19263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:23.129002 19263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "eed5ebe7fbbd47d59d6363787ab24eed"
format_stamp: "Formatted at 2026-08-08 02:22:23 on dist-test-slave-1vmg"
server_key: "8044d04730d2caf2dd3683e1fdb5b093"
server_key_iv: "db860a4a3e63f07f54872c23ad381030"
server_key_version: "encryptionkey@0"
I20260808 02:22:23.129755 19263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "eed5ebe7fbbd47d59d6363787ab24eed"
format_stamp: "Formatted at 2026-08-08 02:22:23 on dist-test-slave-1vmg"
server_key: "8044d04730d2caf2dd3683e1fdb5b093"
server_key_iv: "db860a4a3e63f07f54872c23ad381030"
server_key_version: "encryptionkey@0"
I20260808 02:22:23.135169 19263 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 02:22:23.139007 19288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:23.140834 19263 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:22:23.141011 19263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "eed5ebe7fbbd47d59d6363787ab24eed"
format_stamp: "Formatted at 2026-08-08 02:22:23 on dist-test-slave-1vmg"
server_key: "8044d04730d2caf2dd3683e1fdb5b093"
server_key_iv: "db860a4a3e63f07f54872c23ad381030"
server_key_version: "encryptionkey@0"
I20260808 02:22:23.141155 19263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:23.157014 19263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:23.161849 19263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:23.162107 19263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:23.162890 19263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:22:23.164024 19263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:22:23.164083 19263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:23.164146 19263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:22:23.164178 19263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:23.175714 19263 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:44289
I20260808 02:22:23.175803 19456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:44289 every 8 connection(s)
I20260808 02:22:23.176869 19263 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:22:23.180953 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 19263
I20260808 02:22:23.181118 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:22:23.181429 24895 external_mini_cluster.cc:1428] Setting key aa6efa6d1af8e0d8f71ca9cbd79f9ab9
Aug 08 02:22:23 dist-test-slave-1vmg krb5kdc[18716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155743, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:22:23.195940 18846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:23.179288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:56691 (local address 127.24.79.254:34555)
0808 02:22:23.179453 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:23.179457 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:23.181028 (+ 1571us) server_negotiation.cc:407] Connection header received
0808 02:22:23.182799 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:23.182804 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:23.182874 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:23.183016 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:23.184980 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.185566 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.187240 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.187485 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.187600 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:23.191342 (+ 3742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:23.191346 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:23.191349 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:23.191383 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:23.192954 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:23.193748 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:23.193752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:23.193753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:23.193866 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:23.194341 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:23.194344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:23.194346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:23.194587 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:23.194700 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:23.195546 (+ 846us) server_negotiation.cc:299] Negotiation successful
0808 02:22:23.195740 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 02:22:23.197125 19460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:23.180018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:34555 (local address 127.24.79.195:56691)
0808 02:22:23.180761 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:23.180828 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:23.182487 (+ 1659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:23.183394 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:23.183405 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:23.183983 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:23.184783 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:23.184805 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.185744 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.185747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:23.186984 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:23.187003 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.187719 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.187722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:23.187933 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:23.188959 (+ 1026us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:22:23.188982 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:23.191026 (+ 2044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:23.193199 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:23.193206 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:23.193226 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:23.193572 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:23.193982 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:23.193985 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:23.193986 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:23.194192 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:23.194758 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:23.194766 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:23.195222 (+ 456us) client_negotiation.cc:769] Sending connection context
0808 02:22:23.195463 (+ 241us) client_negotiation.cc:240] Negotiation successful
0808 02:22:23.195911 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":187,"threads_started":1}
I20260808 02:22:23.198796 19457 heartbeater.cc:347] Connected to a master server at 127.24.79.254:34555
I20260808 02:22:23.199136 19457 heartbeater.cc:464] Registering TS with master...
I20260808 02:22:23.199846 19457 heartbeater.cc:511] Master 127.24.79.254:34555 requested a full tablet report, sending...
I20260808 02:22:23.201365 18786 ts_manager.cc:194] Registered new tserver with Master: eed5ebe7fbbd47d59d6363787ab24eed (127.24.79.195:44289)
I20260808 02:22:23.202071 18786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.195@KRBTEST.COM'} at 127.24.79.195:56691
I20260808 02:22:23.210261 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:22:23.223006 18846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:23.212673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48728 (local address 127.24.79.254:34555)
0808 02:22:23.212882 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:23.212886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:23.213097 (+ 211us) server_negotiation.cc:407] Connection header received
0808 02:22:23.213159 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:23.213162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:23.213214 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:23.213278 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:23.214225 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.214789 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.215819 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.216283 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.216416 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:23.218647 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:23.218651 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:23.218654 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:23.218689 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:23.220353 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:23.221041 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:23.221045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:23.221046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:23.221284 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:23.221818 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:23.221821 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:23.221823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:23.222096 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:23.222260 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:23.222660 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 02:22:23.222780 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260808 02:22:23.227156 18786 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:22:23.230594 18786 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:22:23.248823 19476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:23.242158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56184 (local address 127.24.79.194:42927)
0808 02:22:23.242632 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:23.242645 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:23.242674 (+ 29us) server_negotiation.cc:407] Connection header received
0808 02:22:23.242767 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:23.242773 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:23.243069 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:23.243704 (+ 635us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:23.243884 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.245198 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.247104 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.248002 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.248117 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:23.248315 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:23.248539 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 02:22:23.248661 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":155,"threads_started":1}
I20260808 02:22:23.248826 19472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:23.241864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:42927 (local address 127.0.0.1:56184)
0808 02:22:23.242306 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:23.242374 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:23.242548 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:23.243296 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:23.243303 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:23.243331 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:23.243521 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:23.243530 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.245365 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.245369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:23.246882 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:23.246894 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.248153 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.248157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:23.248288 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:23.248336 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 02:22:23.248467 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 02:22:23.248640 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":21,"thread_start_us":89,"threads_started":1}
I20260808 02:22:23.249264 19478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:23.241868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:44289 (local address 127.0.0.1:55400)
0808 02:22:23.243061 (+ 1193us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:23.243075 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:23.243187 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:23.244052 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:23.244055 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:23.244072 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:23.244177 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:23.244183 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.245621 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.245624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:23.247181 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:23.247196 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.248780 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.248784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:23.248924 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:23.248950 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 02:22:23.249022 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 02:22:23.249102 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1124,"mutex_wait_us":562,"thread_start_us":137,"threads_started":1}
I20260808 02:22:23.249697 19477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:23.242777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:37189 (local address 127.0.0.1:55976)
0808 02:22:23.243736 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:23.243750 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:23.243919 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:23.244533 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:23.244535 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:23.244550 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:23.244616 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:23.244619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.245672 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.245675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:23.247185 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:23.247196 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.248780 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.248784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:23.249312 (+ 528us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:23.249402 (+ 90us) client_negotiation.cc:769] Sending connection context
0808 02:22:23.249485 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 02:22:23.249565 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":894,"mutex_wait_us":123,"thread_start_us":101,"threads_started":1}
I20260808 02:22:23.249755 19474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:23.242006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55400 (local address 127.24.79.195:44289)
0808 02:22:23.242827 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:23.242840 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:23.243380 (+ 540us) server_negotiation.cc:407] Connection header received
0808 02:22:23.243485 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:23.243491 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:23.243716 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:23.243880 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:23.244326 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.245448 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.247362 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.248425 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.248551 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:23.248704 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:23.249487 (+ 783us) server_negotiation.cc:299] Negotiation successful
0808 02:22:23.249621 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":142,"threads_started":1}
I20260808 02:22:23.250267 19479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:23.242998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55976 (local address 127.24.79.193:37189)
0808 02:22:23.243565 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:23.243571 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:23.243846 (+ 275us) server_negotiation.cc:407] Connection header received
0808 02:22:23.244069 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:23.244075 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:23.244283 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:23.244386 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:23.244743 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.245557 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.247362 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.248425 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.248543 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:23.248703 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:23.249972 (+ 1269us) server_negotiation.cc:299] Negotiation successful
0808 02:22:23.250104 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":107,"threads_started":1}
I20260808 02:22:23.252739 19359 tablet_service.cc:1467] Processing CreateTablet for tablet 86767f900fb14fb68300556b3525d412 (DEFAULT_TABLE table=test-table [id=8e6bdb02817b4a9e8a1490adf71e6a1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:22:23.252753 19165 tablet_service.cc:1467] Processing CreateTablet for tablet 86767f900fb14fb68300556b3525d412 (DEFAULT_TABLE table=test-table [id=8e6bdb02817b4a9e8a1490adf71e6a1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:22:23.253132 18969 tablet_service.cc:1467] Processing CreateTablet for tablet 86767f900fb14fb68300556b3525d412 (DEFAULT_TABLE table=test-table [id=8e6bdb02817b4a9e8a1490adf71e6a1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:22:23.254074 19165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86767f900fb14fb68300556b3525d412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:23.254074 18969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86767f900fb14fb68300556b3525d412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:23.254076 19359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86767f900fb14fb68300556b3525d412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:23.262468 19483 tablet_bootstrap.cc:492] T 86767f900fb14fb68300556b3525d412 P e271b6d6febe4d26b315ab8d16296274: Bootstrap starting.
I20260808 02:22:23.263193 19481 tablet_bootstrap.cc:492] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed: Bootstrap starting.
I20260808 02:22:23.265723 19481 tablet_bootstrap.cc:654] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:23.267627 19481 log.cc:824] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:23.267858 19482 tablet_bootstrap.cc:492] T 86767f900fb14fb68300556b3525d412 P c12b99125a4f4a998260914679cf4039: Bootstrap starting.
I20260808 02:22:23.269136 19483 tablet_bootstrap.cc:654] T 86767f900fb14fb68300556b3525d412 P e271b6d6febe4d26b315ab8d16296274: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:23.270372 19483 log.cc:824] T 86767f900fb14fb68300556b3525d412 P e271b6d6febe4d26b315ab8d16296274: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:23.270531 19481 tablet_bootstrap.cc:492] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed: No bootstrap required, opened a new log
I20260808 02:22:23.270637 19482 tablet_bootstrap.cc:654] T 86767f900fb14fb68300556b3525d412 P c12b99125a4f4a998260914679cf4039: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:23.270817 19481 ts_tablet_manager.cc:1397] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260808 02:22:23.272149 19482 log.cc:824] T 86767f900fb14fb68300556b3525d412 P c12b99125a4f4a998260914679cf4039: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:23.272800 19483 tablet_bootstrap.cc:492] T 86767f900fb14fb68300556b3525d412 P e271b6d6febe4d26b315ab8d16296274: No bootstrap required, opened a new log
I20260808 02:22:23.273064 19483 ts_tablet_manager.cc:1397] T 86767f900fb14fb68300556b3525d412 P e271b6d6febe4d26b315ab8d16296274: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260808 02:22:23.274827 19482 tablet_bootstrap.cc:492] T 86767f900fb14fb68300556b3525d412 P c12b99125a4f4a998260914679cf4039: No bootstrap required, opened a new log
I20260808 02:22:23.274734 19481 raft_consensus.cc:348] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed5ebe7fbbd47d59d6363787ab24eed" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44289 } } peers { permanent_uuid: "c12b99125a4f4a998260914679cf4039" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 42927 } } peers { permanent_uuid: "e271b6d6febe4d26b315ab8d16296274" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 37189 } }
I20260808 02:22:23.275117 19481 raft_consensus.cc:374] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:23.275156 19482 ts_tablet_manager.cc:1397] T 86767f900fb14fb68300556b3525d412 P c12b99125a4f4a998260914679cf4039: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.004s
I20260808 02:22:23.275179 19481 raft_consensus.cc:729] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eed5ebe7fbbd47d59d6363787ab24eed, State: Initialized, Role: FOLLOWER
I20260808 02:22:23.275844 19481 consensus_queue.cc:260] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed5ebe7fbbd47d59d6363787ab24eed" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44289 } } peers { permanent_uuid: "c12b99125a4f4a998260914679cf4039" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 42927 } } peers { permanent_uuid: "e271b6d6febe4d26b315ab8d16296274" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 37189 } }
I20260808 02:22:23.276877 19457 heartbeater.cc:503] Master 127.24.79.254:34555 was elected leader, sending a full tablet report...
I20260808 02:22:23.277169 19481 ts_tablet_manager.cc:1428] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260808 02:22:23.278230 19483 raft_consensus.cc:348] T 86767f900fb14fb68300556b3525d412 P e271b6d6febe4d26b315ab8d16296274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed5ebe7fbbd47d59d6363787ab24eed" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44289 } } peers { permanent_uuid: "c12b99125a4f4a998260914679cf4039" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 42927 } } peers { permanent_uuid: "e271b6d6febe4d26b315ab8d16296274" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 37189 } }
I20260808 02:22:23.278616 19483 raft_consensus.cc:374] T 86767f900fb14fb68300556b3525d412 P e271b6d6febe4d26b315ab8d16296274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:23.278679 19483 raft_consensus.cc:729] T 86767f900fb14fb68300556b3525d412 P e271b6d6febe4d26b315ab8d16296274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e271b6d6febe4d26b315ab8d16296274, State: Initialized, Role: FOLLOWER
I20260808 02:22:23.279284 19483 consensus_queue.cc:260] T 86767f900fb14fb68300556b3525d412 P e271b6d6febe4d26b315ab8d16296274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed5ebe7fbbd47d59d6363787ab24eed" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44289 } } peers { permanent_uuid: "c12b99125a4f4a998260914679cf4039" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 42927 } } peers { permanent_uuid: "e271b6d6febe4d26b315ab8d16296274" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 37189 } }
I20260808 02:22:23.279458 19482 raft_consensus.cc:348] T 86767f900fb14fb68300556b3525d412 P c12b99125a4f4a998260914679cf4039 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed5ebe7fbbd47d59d6363787ab24eed" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44289 } } peers { permanent_uuid: "c12b99125a4f4a998260914679cf4039" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 42927 } } peers { permanent_uuid: "e271b6d6febe4d26b315ab8d16296274" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 37189 } }
I20260808 02:22:23.279820 19482 raft_consensus.cc:374] T 86767f900fb14fb68300556b3525d412 P c12b99125a4f4a998260914679cf4039 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:23.279933 19482 raft_consensus.cc:729] T 86767f900fb14fb68300556b3525d412 P c12b99125a4f4a998260914679cf4039 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c12b99125a4f4a998260914679cf4039, State: Initialized, Role: FOLLOWER
I20260808 02:22:23.280486 19063 heartbeater.cc:503] Master 127.24.79.254:34555 was elected leader, sending a full tablet report...
I20260808 02:22:23.280658 19483 ts_tablet_manager.cc:1428] T 86767f900fb14fb68300556b3525d412 P e271b6d6febe4d26b315ab8d16296274: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260808 02:22:23.281060 19482 consensus_queue.cc:260] T 86767f900fb14fb68300556b3525d412 P c12b99125a4f4a998260914679cf4039 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed5ebe7fbbd47d59d6363787ab24eed" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44289 } } peers { permanent_uuid: "c12b99125a4f4a998260914679cf4039" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 42927 } } peers { permanent_uuid: "e271b6d6febe4d26b315ab8d16296274" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 37189 } }
I20260808 02:22:23.282071 19258 heartbeater.cc:503] Master 127.24.79.254:34555 was elected leader, sending a full tablet report...
I20260808 02:22:23.282418 19482 ts_tablet_manager.cc:1428] T 86767f900fb14fb68300556b3525d412 P c12b99125a4f4a998260914679cf4039: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260808 02:22:23.400893 19064 tablet.cc:2369] T 86767f900fb14fb68300556b3525d412 P e271b6d6febe4d26b315ab8d16296274: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:22:23.407173 19259 tablet.cc:2369] T 86767f900fb14fb68300556b3525d412 P c12b99125a4f4a998260914679cf4039: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:22:23.428289 19459 tablet.cc:2369] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:22:23.442492 19488 raft_consensus.cc:482] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:22:23.442741 19488 raft_consensus.cc:504] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed5ebe7fbbd47d59d6363787ab24eed" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44289 } } peers { permanent_uuid: "c12b99125a4f4a998260914679cf4039" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 42927 } } peers { permanent_uuid: "e271b6d6febe4d26b315ab8d16296274" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 37189 } }
I20260808 02:22:23.444072 19488 leader_election.cc:288] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c12b99125a4f4a998260914679cf4039 (127.24.79.194:42927), e271b6d6febe4d26b315ab8d16296274 (127.24.79.193:37189)
I20260808 02:22:23.448293 19460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:23.444424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:42927 (local address 127.24.79.195:46713)
0808 02:22:23.444645 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:23.444663 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:23.444794 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:23.445110 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:23.445114 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:23.445182 (+ 68us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:23.445315 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:23.445323 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.446219 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.446222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:23.447157 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:23.447169 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.447821 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.447825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:23.447941 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:23.447967 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 02:22:23.448045 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 02:22:23.448135 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 02:22:23.448293 19476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:23.444598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:46713 (local address 127.24.79.194:42927)
0808 02:22:23.444786 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:23.444791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:23.444809 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:22:23.444905 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:23.444909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:23.444960 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:23.445040 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:23.445452 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.446102 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.447289 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.447702 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.447821 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:23.447883 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:23.448068 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 02:22:23.448150 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260808 02:22:23.448472 19492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:23.444744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:37189 (local address 127.24.79.195:40565)
0808 02:22:23.445107 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:23.445123 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:23.445308 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:23.445560 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:23.445563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:23.445581 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:23.445717 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:23.445725 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.446494 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.446498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:23.447344 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:23.447354 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.448042 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.448046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:23.448153 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:23.448175 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 02:22:23.448240 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 02:22:23.448313 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":149,"threads_started":1}
I20260808 02:22:23.448462 19479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:23.444920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:40565 (local address 127.24.79.193:37189)
0808 02:22:23.445056 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:23.445061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:23.445270 (+ 209us) server_negotiation.cc:407] Connection header received
0808 02:22:23.445410 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:23.445413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:23.445469 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:23.445529 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:23.445815 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.446370 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.447486 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.447919 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.448032 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:23.448087 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:23.448268 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 02:22:23.448352 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 02:22:23.449126 19002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86767f900fb14fb68300556b3525d412" candidate_uuid: "eed5ebe7fbbd47d59d6363787ab24eed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e271b6d6febe4d26b315ab8d16296274" is_pre_election: true
I20260808 02:22:23.449138 19193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86767f900fb14fb68300556b3525d412" candidate_uuid: "eed5ebe7fbbd47d59d6363787ab24eed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c12b99125a4f4a998260914679cf4039" is_pre_election: true
I20260808 02:22:23.449496 19002 raft_consensus.cc:2454] T 86767f900fb14fb68300556b3525d412 P e271b6d6febe4d26b315ab8d16296274 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eed5ebe7fbbd47d59d6363787ab24eed in term 0.
I20260808 02:22:23.449496 19193 raft_consensus.cc:2454] T 86767f900fb14fb68300556b3525d412 P c12b99125a4f4a998260914679cf4039 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eed5ebe7fbbd47d59d6363787ab24eed in term 0.
I20260808 02:22:23.450031 19298 leader_election.cc:302] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e271b6d6febe4d26b315ab8d16296274, eed5ebe7fbbd47d59d6363787ab24eed; no voters:
I20260808 02:22:23.450357 19488 raft_consensus.cc:2781] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:22:23.450464 19488 raft_consensus.cc:482] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:22:23.450513 19488 raft_consensus.cc:3037] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:23.451823 19488 raft_consensus.cc:504] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed5ebe7fbbd47d59d6363787ab24eed" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44289 } } peers { permanent_uuid: "c12b99125a4f4a998260914679cf4039" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 42927 } } peers { permanent_uuid: "e271b6d6febe4d26b315ab8d16296274" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 37189 } }
I20260808 02:22:23.452235 19488 leader_election.cc:288] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed [CANDIDATE]: Term 1 election: Requested vote from peers c12b99125a4f4a998260914679cf4039 (127.24.79.194:42927), e271b6d6febe4d26b315ab8d16296274 (127.24.79.193:37189)
I20260808 02:22:23.452855 19193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86767f900fb14fb68300556b3525d412" candidate_uuid: "eed5ebe7fbbd47d59d6363787ab24eed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c12b99125a4f4a998260914679cf4039"
I20260808 02:22:23.453004 19193 raft_consensus.cc:3037] T 86767f900fb14fb68300556b3525d412 P c12b99125a4f4a998260914679cf4039 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:23.453002 19002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86767f900fb14fb68300556b3525d412" candidate_uuid: "eed5ebe7fbbd47d59d6363787ab24eed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e271b6d6febe4d26b315ab8d16296274"
I20260808 02:22:23.453167 19002 raft_consensus.cc:3037] T 86767f900fb14fb68300556b3525d412 P e271b6d6febe4d26b315ab8d16296274 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:23.454360 19193 raft_consensus.cc:2454] T 86767f900fb14fb68300556b3525d412 P c12b99125a4f4a998260914679cf4039 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eed5ebe7fbbd47d59d6363787ab24eed in term 1.
I20260808 02:22:23.454380 19002 raft_consensus.cc:2454] T 86767f900fb14fb68300556b3525d412 P e271b6d6febe4d26b315ab8d16296274 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eed5ebe7fbbd47d59d6363787ab24eed in term 1.
I20260808 02:22:23.454767 19298 leader_election.cc:302] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c12b99125a4f4a998260914679cf4039, eed5ebe7fbbd47d59d6363787ab24eed; no voters:
I20260808 02:22:23.454991 19488 raft_consensus.cc:2781] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:23.455266 19488 raft_consensus.cc:686] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed [term 1 LEADER]: Becoming Leader. State: Replica: eed5ebe7fbbd47d59d6363787ab24eed, State: Running, Role: LEADER
I20260808 02:22:23.455665 19488 consensus_queue.cc:237] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed5ebe7fbbd47d59d6363787ab24eed" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44289 } } peers { permanent_uuid: "c12b99125a4f4a998260914679cf4039" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 42927 } } peers { permanent_uuid: "e271b6d6febe4d26b315ab8d16296274" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 37189 } }
I20260808 02:22:23.459559 18786 catalog_manager.cc:5495] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed reported cstate change: term changed from 0 to 1, leader changed from <none> to eed5ebe7fbbd47d59d6363787ab24eed (127.24.79.195). New cstate: current_term: 1 leader_uuid: "eed5ebe7fbbd47d59d6363787ab24eed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed5ebe7fbbd47d59d6363787ab24eed" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 44289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c12b99125a4f4a998260914679cf4039" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 42927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e271b6d6febe4d26b315ab8d16296274" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 37189 } health_report { overall_health: UNKNOWN } } }
I20260808 02:22:23.495699 19474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:23.491738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55404 (local address 127.24.79.195:44289)
0808 02:22:23.491892 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:23.491896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:23.491912 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:22:23.492074 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:23.492077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:23.492139 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:23.492293 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:22:23.492558 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.493073 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.494024 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.494255 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.494347 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:23.494705 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:22:23.495214 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:22:23.495358 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:23.495506 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 02:22:23.495554 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:22:23.502949 19002 raft_consensus.cc:1261] T 86767f900fb14fb68300556b3525d412 P e271b6d6febe4d26b315ab8d16296274 [term 1 FOLLOWER]: Refusing update from remote peer eed5ebe7fbbd47d59d6363787ab24eed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:22:23.502949 19193 raft_consensus.cc:1261] T 86767f900fb14fb68300556b3525d412 P c12b99125a4f4a998260914679cf4039 [term 1 FOLLOWER]: Refusing update from remote peer eed5ebe7fbbd47d59d6363787ab24eed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:22:23.504045 19488 consensus_queue.cc:1035] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed [LEADER]: Connected to new peer: Peer: permanent_uuid: "c12b99125a4f4a998260914679cf4039" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 42927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:22:23.504339 19493 consensus_queue.cc:1035] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed [LEADER]: Connected to new peer: Peer: permanent_uuid: "e271b6d6febe4d26b315ab8d16296274" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 37189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:22:23.528043 18786 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:22:23.528260 18786 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:22:23.531579 18786 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 117e5d13d0b14ae09ed4d89f0c0a13ff: Sending DeleteTablet for 3 replicas of tablet 86767f900fb14fb68300556b3525d412
I20260808 02:22:23.532835 19165 tablet_service.cc:1514] Processing DeleteTablet for tablet 86767f900fb14fb68300556b3525d412 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:22:23 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:56184
I20260808 02:22:23.532975 18969 tablet_service.cc:1514] Processing DeleteTablet for tablet 86767f900fb14fb68300556b3525d412 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:22:23 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:55976
I20260808 02:22:23.533428 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 18881
I20260808 02:22:23.536038 19359 tablet_service.cc:1514] Processing DeleteTablet for tablet 86767f900fb14fb68300556b3525d412 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:22:23 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:55400
I20260808 02:22:23.539510 19516 tablet_replica.cc:330] T 86767f900fb14fb68300556b3525d412 P c12b99125a4f4a998260914679cf4039: stopping tablet replica
I20260808 02:22:23.539973 19516 raft_consensus.cc:2229] T 86767f900fb14fb68300556b3525d412 P c12b99125a4f4a998260914679cf4039 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:22:23.540338 19516 raft_consensus.cc:2258] T 86767f900fb14fb68300556b3525d412 P c12b99125a4f4a998260914679cf4039 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:22:23.548044 19517 tablet_replica.cc:330] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed: stopping tablet replica
I20260808 02:22:23.548501 19517 raft_consensus.cc:2229] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:22:23.549055 19517 raft_consensus.cc:2258] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:22:23.549207 19516 ts_tablet_manager.cc:1905] T 86767f900fb14fb68300556b3525d412 P c12b99125a4f4a998260914679cf4039: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:22:23.552675 19517 ts_tablet_manager.cc:1905] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:22:23.553153 19516 ts_tablet_manager.cc:1918] T 86767f900fb14fb68300556b3525d412 P c12b99125a4f4a998260914679cf4039: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:22:23.553290 19516 log.cc:1196] T 86767f900fb14fb68300556b3525d412 P c12b99125a4f4a998260914679cf4039: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/86767f900fb14fb68300556b3525d412
I20260808 02:22:23.553634 19516 ts_tablet_manager.cc:1939] T 86767f900fb14fb68300556b3525d412 P c12b99125a4f4a998260914679cf4039: Deleting consensus metadata
I20260808 02:22:23.555563 18764 catalog_manager.cc:4841] TS c12b99125a4f4a998260914679cf4039 (127.24.79.194:42927): tablet 86767f900fb14fb68300556b3525d412 (table test-table [id=8e6bdb02817b4a9e8a1490adf71e6a1f]) successfully deleted
I20260808 02:22:23.556335 19517 ts_tablet_manager.cc:1918] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:22:23.556450 19517 log.cc:1196] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/86767f900fb14fb68300556b3525d412
I20260808 02:22:23.556756 19517 ts_tablet_manager.cc:1939] T 86767f900fb14fb68300556b3525d412 P eed5ebe7fbbd47d59d6363787ab24eed: Deleting consensus metadata
I20260808 02:22:23.558344 18759 catalog_manager.cc:4841] TS eed5ebe7fbbd47d59d6363787ab24eed (127.24.79.195:44289): tablet 86767f900fb14fb68300556b3525d412 (table test-table [id=8e6bdb02817b4a9e8a1490adf71e6a1f]) successfully deleted
W20260808 02:22:23.560209 18764 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.79.193:37189 (error 108)
I20260808 02:22:23.561257 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 19074
I20260808 02:22:23.561471 19477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:23.561134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:37189 (local address 127.0.0.1:55982)
0808 02:22:23.561302 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:23.561378 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.193:37189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260808 02:22:23.572745 18764 catalog_manager.cc:4584] TS e271b6d6febe4d26b315ab8d16296274 (127.24.79.193:37189): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 86767f900fb14fb68300556b3525d412: Network error: Client connection negotiation failed: client connection to 127.24.79.193:37189: connect: Connection refused (error 111)
I20260808 02:22:23.580363 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 19263
I20260808 02:22:23.591500 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 18735
2026-08-08T02:22:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2833 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T02:22:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:23Z Disabled control of system clock
I20260808 02:22:23.639468 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:41099
--webserver_interface=127.24.79.254
--webserver_port=0
--builtin_ntp_servers=127.24.79.212:43251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:41099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:22:23.766084 19529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:23.766419 19529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:23.766456 19529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:23.770709 19529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:23.770808 19529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:23.770823 19529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:23.770838 19529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:23.770853 19529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:23.776467 19529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:43251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:41099
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:41099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:23.777765 19529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:23.779779 19529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:23.789350 19537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:23.789350 19535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:23.789412 19540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:23.789762 19529 server_base.cc:1034] running on GCE node
I20260808 02:22:23.790562 19529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:23.791961 19529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:23.793172 19529 hybrid_clock.cc:648] HybridClock initialized: now 1786155743793146 us; error 50 us; skew 500 ppm
I20260808 02:22:23.795549 19529 webserver.cc:460] Webserver started at http://127.24.79.254:34637/ using document root <none> and password file <none>
I20260808 02:22:23.796200 19529 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:23.796274 19529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:23.796504 19529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:23.798895 19529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "6a21f756013a4965aec5d56f3ffc580e"
format_stamp: "Formatted at 2026-08-08 02:22:23 on dist-test-slave-1vmg"
server_key: "8a1e96ee6fc7b4aef43d0578c59fa17c"
server_key_iv: "9f088071aebb96ef1855397192c23a9d"
server_key_version: "encryptionkey@0"
I20260808 02:22:23.799574 19529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "6a21f756013a4965aec5d56f3ffc580e"
format_stamp: "Formatted at 2026-08-08 02:22:23 on dist-test-slave-1vmg"
server_key: "8a1e96ee6fc7b4aef43d0578c59fa17c"
server_key_iv: "9f088071aebb96ef1855397192c23a9d"
server_key_version: "encryptionkey@0"
I20260808 02:22:23.803969 19529 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:22:23.807451 19547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:23.809417 19529 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:22:23.809628 19529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "6a21f756013a4965aec5d56f3ffc580e"
format_stamp: "Formatted at 2026-08-08 02:22:23 on dist-test-slave-1vmg"
server_key: "8a1e96ee6fc7b4aef43d0578c59fa17c"
server_key_iv: "9f088071aebb96ef1855397192c23a9d"
server_key_version: "encryptionkey@0"
I20260808 02:22:23.809790 19529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:23.823434 19529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:23.824297 19529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:23.824510 19529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:23.833324 19529 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:41099
I20260808 02:22:23.833436 19630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:41099 every 8 connection(s)
I20260808 02:22:23.834497 19529 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:23.837610 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 19529
I20260808 02:22:23.837744 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:23.838048 24895 external_mini_cluster.cc:1428] Setting key a034bcc445ed9e84de172f52efb58b56
I20260808 02:22:23.838783 19631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:23.847249 19631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a21f756013a4965aec5d56f3ffc580e: Bootstrap starting.
I20260808 02:22:23.849799 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:23.839744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38804 (local address 127.24.79.254:41099)
0808 02:22:23.840604 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:23.840628 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:23.840684 (+ 56us) server_negotiation.cc:407] Connection header received
0808 02:22:23.841447 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:23.841471 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:23.841962 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:23.842327 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:23.842846 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.844194 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.845617 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:23.846238 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:23.846394 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:23.847567 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:23.847573 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:23.847591 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:23.848100 (+ 509us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:23.848181 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:23.848208 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:23.848297 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:23.848505 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 02:22:23.848883 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":168,"threads_started":1}
I20260808 02:22:23.850117 19631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a21f756013a4965aec5d56f3ffc580e: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:23.851053 19631 log.cc:824] T 00000000000000000000000000000000 P 6a21f756013a4965aec5d56f3ffc580e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:23.853581 19631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a21f756013a4965aec5d56f3ffc580e: No bootstrap required, opened a new log
I20260808 02:22:23.857515 19631 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a21f756013a4965aec5d56f3ffc580e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a21f756013a4965aec5d56f3ffc580e" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41099 } }
I20260808 02:22:23.857926 19631 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a21f756013a4965aec5d56f3ffc580e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:23.858004 19631 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a21f756013a4965aec5d56f3ffc580e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a21f756013a4965aec5d56f3ffc580e, State: Initialized, Role: FOLLOWER
I20260808 02:22:23.858741 19631 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a21f756013a4965aec5d56f3ffc580e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a21f756013a4965aec5d56f3ffc580e" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41099 } }
I20260808 02:22:23.858959 19631 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a21f756013a4965aec5d56f3ffc580e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:23.859033 19631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a21f756013a4965aec5d56f3ffc580e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:23.859127 19631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a21f756013a4965aec5d56f3ffc580e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:23.860256 19631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a21f756013a4965aec5d56f3ffc580e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a21f756013a4965aec5d56f3ffc580e" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41099 } }
I20260808 02:22:23.860661 19631 leader_election.cc:302] T 00000000000000000000000000000000 P 6a21f756013a4965aec5d56f3ffc580e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a21f756013a4965aec5d56f3ffc580e; no voters:
I20260808 02:22:23.861003 19631 leader_election.cc:288] T 00000000000000000000000000000000 P 6a21f756013a4965aec5d56f3ffc580e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:22:23.861228 19639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a21f756013a4965aec5d56f3ffc580e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:23.861464 19639 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a21f756013a4965aec5d56f3ffc580e [term 1 LEADER]: Becoming Leader. State: Replica: 6a21f756013a4965aec5d56f3ffc580e, State: Running, Role: LEADER
I20260808 02:22:23.861881 19639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a21f756013a4965aec5d56f3ffc580e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a21f756013a4965aec5d56f3ffc580e" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41099 } }
I20260808 02:22:23.862466 19631 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a21f756013a4965aec5d56f3ffc580e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:23.863667 19640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a21f756013a4965aec5d56f3ffc580e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a21f756013a4965aec5d56f3ffc580e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a21f756013a4965aec5d56f3ffc580e" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41099 } } }
I20260808 02:22:23.863822 19640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a21f756013a4965aec5d56f3ffc580e [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:23.864142 19641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a21f756013a4965aec5d56f3ffc580e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a21f756013a4965aec5d56f3ffc580e. Latest consensus state: current_term: 1 leader_uuid: "6a21f756013a4965aec5d56f3ffc580e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a21f756013a4965aec5d56f3ffc580e" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41099 } } }
I20260808 02:22:23.864243 19641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a21f756013a4965aec5d56f3ffc580e [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:23.864348 19650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:22:23.867976 19650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:22:23.875319 19650 catalog_manager.cc:1347] Generated new cluster ID: 47c71d2d6b85436b9fc627f153fcdafa
I20260808 02:22:23.875413 19650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:22:23.894321 19650 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:22:23.895831 19650 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:22:23.914318 19650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a21f756013a4965aec5d56f3ffc580e: Generated new TSK 0
I20260808 02:22:23.915472 19650 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:22:23.921546 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:41099
--builtin_ntp_servers=127.24.79.212:43251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:22:24.058177 19664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:24.058519 19664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:24.058560 19664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:24.062853 19664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:24.062997 19664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:24.063081 19664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:22:24.068392 19664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:43251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:41099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:24.069701 19664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:24.071317 19664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:24.083180 19676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:24.083180 19673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:24.083312 19664 server_base.cc:1034] running on GCE node
W20260808 02:22:24.083254 19674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:24.083994 19664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:24.084792 19664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:24.086009 19664 hybrid_clock.cc:648] HybridClock initialized: now 1786155744085999 us; error 54 us; skew 500 ppm
I20260808 02:22:24.088577 19664 webserver.cc:460] Webserver started at http://127.24.79.193:36659/ using document root <none> and password file <none>
I20260808 02:22:24.089294 19664 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:24.089377 19664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:24.089612 19664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:24.092173 19664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "97ed2c344a324a67958fa48707a324d6"
format_stamp: "Formatted at 2026-08-08 02:22:24 on dist-test-slave-1vmg"
server_key: "e286696987d20af020520373f2dee8cb"
server_key_iv: "797d1fe997eb530527041a0549e964a1"
server_key_version: "encryptionkey@0"
I20260808 02:22:24.092854 19664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "97ed2c344a324a67958fa48707a324d6"
format_stamp: "Formatted at 2026-08-08 02:22:24 on dist-test-slave-1vmg"
server_key: "e286696987d20af020520373f2dee8cb"
server_key_iv: "797d1fe997eb530527041a0549e964a1"
server_key_version: "encryptionkey@0"
I20260808 02:22:24.097434 19664 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:22:24.100622 19689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:24.102411 19664 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:22:24.102584 19664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "97ed2c344a324a67958fa48707a324d6"
format_stamp: "Formatted at 2026-08-08 02:22:24 on dist-test-slave-1vmg"
server_key: "e286696987d20af020520373f2dee8cb"
server_key_iv: "797d1fe997eb530527041a0549e964a1"
server_key_version: "encryptionkey@0"
I20260808 02:22:24.102733 19664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:24.119846 19664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:24.120719 19664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:24.120944 19664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:24.121661 19664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:22:24.122807 19664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:22:24.122864 19664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:24.122949 19664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:22:24.122982 19664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:24.133641 19664 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:37507
I20260808 02:22:24.133826 19868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:37507 every 8 connection(s)
I20260808 02:22:24.134902 19664 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:22:24.141364 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 19664
I20260808 02:22:24.141530 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:22:24.141880 24895 external_mini_cluster.cc:1428] Setting key c8ac4343adf820da0a782959d8f4c2e1
I20260808 02:22:24.144685 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:41099
--builtin_ntp_servers=127.24.79.212:43251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 02:22:24.147872 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:24.137424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:52841 (local address 127.24.79.254:41099)
0808 02:22:24.137655 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:24.137660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:24.139059 (+ 1399us) server_negotiation.cc:407] Connection header received
0808 02:22:24.140694 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:24.140698 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:24.140765 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:24.140869 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:24.142521 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.143220 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.145286 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.145546 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.145657 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:24.147202 (+ 1545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:24.147207 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:24.147210 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:24.147231 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:24.147259 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:24.147271 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:24.147360 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:24.147610 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 02:22:24.147703 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260808 02:22:24.149189 19872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:24.137888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:41099 (local address 127.24.79.193:52841)
0808 02:22:24.138796 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:24.138849 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:24.140348 (+ 1499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:24.141227 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:24.141241 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:24.141387 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:24.142271 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:24.142298 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.143429 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.143433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:24.145031 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:24.145050 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.145763 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.145767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:24.146004 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:24.146800 (+ 796us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:22:24.146802 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:24.146857 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:22:24.146987 (+ 130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:24.147412 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:24.147425 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:24.147432 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 02:22:24.147614 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 02:22:24.147951 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":167,"threads_started":1}
I20260808 02:22:24.151060 19869 heartbeater.cc:347] Connected to a master server at 127.24.79.254:41099
I20260808 02:22:24.151432 19869 heartbeater.cc:464] Registering TS with master...
I20260808 02:22:24.152107 19869 heartbeater.cc:511] Master 127.24.79.254:41099 requested a full tablet report, sending...
I20260808 02:22:24.154546 19562 ts_manager.cc:194] Registered new tserver with Master: 97ed2c344a324a67958fa48707a324d6 (127.24.79.193:37507)
I20260808 02:22:24.156059 19562 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.79.193:52841
W20260808 02:22:24.278172 19873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:24.278548 19873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:24.278589 19873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:24.282884 19873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:24.283011 19873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:24.283073 19873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:22:24.288204 19873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:43251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:41099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:24.289507 19873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:24.291348 19873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:24.303000 19884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:24.302946 19883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:24.303253 19873 server_base.cc:1034] running on GCE node
W20260808 02:22:24.302942 19886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:24.303897 19873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:24.304649 19873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:24.305882 19873 hybrid_clock.cc:648] HybridClock initialized: now 1786155744305859 us; error 42 us; skew 500 ppm
I20260808 02:22:24.308394 19873 webserver.cc:460] Webserver started at http://127.24.79.194:39525/ using document root <none> and password file <none>
I20260808 02:22:24.309104 19873 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:24.309192 19873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:24.309422 19873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:24.312156 19873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "4eabec7a8a2743928f4a89c5afbc7322"
format_stamp: "Formatted at 2026-08-08 02:22:24 on dist-test-slave-1vmg"
server_key: "7e8200c77454cd5cf90d1b58bc8ddac9"
server_key_iv: "f6c105d22f11c94de03ffb0d25d58b54"
server_key_version: "encryptionkey@0"
I20260808 02:22:24.312870 19873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "4eabec7a8a2743928f4a89c5afbc7322"
format_stamp: "Formatted at 2026-08-08 02:22:24 on dist-test-slave-1vmg"
server_key: "7e8200c77454cd5cf90d1b58bc8ddac9"
server_key_iv: "f6c105d22f11c94de03ffb0d25d58b54"
server_key_version: "encryptionkey@0"
I20260808 02:22:24.317736 19873 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:22:24.320829 19893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:24.322501 19873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:22:24.322675 19873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "4eabec7a8a2743928f4a89c5afbc7322"
format_stamp: "Formatted at 2026-08-08 02:22:24 on dist-test-slave-1vmg"
server_key: "7e8200c77454cd5cf90d1b58bc8ddac9"
server_key_iv: "f6c105d22f11c94de03ffb0d25d58b54"
server_key_version: "encryptionkey@0"
I20260808 02:22:24.322824 19873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:24.346505 19873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:24.347555 19873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:24.347841 19873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:24.348658 19873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:22:24.349906 19873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:22:24.349974 19873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:24.350056 19873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:22:24.350088 19873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:24.360184 19873 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:37061
I20260808 02:22:24.360260 20060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:37061 every 8 connection(s)
I20260808 02:22:24.361325 19873 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:22:24.365023 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 19873
I20260808 02:22:24.365167 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:22:24.365473 24895 external_mini_cluster.cc:1428] Setting key 54a82aed5e7ee776d327317296a7f0e3
I20260808 02:22:24.367821 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:41099
--builtin_ntp_servers=127.24.79.212:43251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 02:22:24.373881 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:24.363649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:42969 (local address 127.24.79.254:41099)
0808 02:22:24.363784 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:24.363789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:24.365472 (+ 1683us) server_negotiation.cc:407] Connection header received
0808 02:22:24.367223 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:24.367227 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:24.367300 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:24.367378 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:24.368563 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.369190 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.371255 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.371495 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.371596 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:24.373244 (+ 1648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:24.373248 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:24.373250 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:24.373268 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:24.373292 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:24.373301 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:24.373372 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:24.373652 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 02:22:24.373741 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 02:22:24.375380 20063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:24.364214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:41099 (local address 127.24.79.194:42969)
0808 02:22:24.365281 (+ 1067us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:24.365339 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:24.366982 (+ 1643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:24.367611 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:24.367621 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:24.367728 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:24.368368 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:24.368386 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.369351 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.369354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:24.371059 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:24.371077 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.371716 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.371721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:24.371951 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:24.372807 (+ 856us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:22:24.372809 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:24.372877 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:22:24.372984 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:24.373407 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:24.373422 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:24.373429 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 02:22:24.373594 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 02:22:24.374007 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":190,"threads_started":1}
I20260808 02:22:24.377444 20061 heartbeater.cc:347] Connected to a master server at 127.24.79.254:41099
I20260808 02:22:24.377763 20061 heartbeater.cc:464] Registering TS with master...
I20260808 02:22:24.378423 20061 heartbeater.cc:511] Master 127.24.79.254:41099 requested a full tablet report, sending...
I20260808 02:22:24.379842 19562 ts_manager.cc:194] Registered new tserver with Master: 4eabec7a8a2743928f4a89c5afbc7322 (127.24.79.194:37061)
I20260808 02:22:24.380506 19562 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.79.194:42969
W20260808 02:22:24.495543 20064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:24.495867 20064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:24.495908 20064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:24.500406 20064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:24.500489 20064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:24.500551 20064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:22:24.505896 20064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:43251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:41099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:24.507316 20064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:24.509037 20064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:24.520644 20073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:24.520692 20076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:24.520709 20072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:24.520975 20064 server_base.cc:1034] running on GCE node
I20260808 02:22:24.521479 20064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:24.522282 20064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:24.523491 20064 hybrid_clock.cc:648] HybridClock initialized: now 1786155744523450 us; error 64 us; skew 500 ppm
I20260808 02:22:24.526033 20064 webserver.cc:460] Webserver started at http://127.24.79.195:41369/ using document root <none> and password file <none>
I20260808 02:22:24.526786 20064 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:24.526872 20064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:24.527155 20064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:24.529621 20064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "d8905291085d4821b88165085e9ce4f7"
format_stamp: "Formatted at 2026-08-08 02:22:24 on dist-test-slave-1vmg"
server_key: "5518c5c585fed4d25bfb96762858cdee"
server_key_iv: "8a2388daf6d1f13c13ffa60d263d76bd"
server_key_version: "encryptionkey@0"
I20260808 02:22:24.530298 20064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "d8905291085d4821b88165085e9ce4f7"
format_stamp: "Formatted at 2026-08-08 02:22:24 on dist-test-slave-1vmg"
server_key: "5518c5c585fed4d25bfb96762858cdee"
server_key_iv: "8a2388daf6d1f13c13ffa60d263d76bd"
server_key_version: "encryptionkey@0"
I20260808 02:22:24.534592 20064 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:22:24.537708 20086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:24.539340 20064 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:22:24.539515 20064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "d8905291085d4821b88165085e9ce4f7"
format_stamp: "Formatted at 2026-08-08 02:22:24 on dist-test-slave-1vmg"
server_key: "5518c5c585fed4d25bfb96762858cdee"
server_key_iv: "8a2388daf6d1f13c13ffa60d263d76bd"
server_key_version: "encryptionkey@0"
I20260808 02:22:24.539650 20064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:24.571935 20064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:24.572840 20064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:24.573079 20064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:24.573801 20064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:22:24.574991 20064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:22:24.575052 20064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:24.575130 20064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:22:24.575157 20064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:24.585438 20064 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:45887
I20260808 02:22:24.585462 20266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:45887 every 8 connection(s)
I20260808 02:22:24.586508 20064 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:22:24.588382 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 20064
I20260808 02:22:24.588513 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:22:24.588860 24895 external_mini_cluster.cc:1428] Setting key 7f32efefafd4fef871d1bc5c0272e7c4
I20260808 02:22:24.598014 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:24.588775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:43485 (local address 127.24.79.254:41099)
0808 02:22:24.588977 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:24.588983 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:24.590486 (+ 1503us) server_negotiation.cc:407] Connection header received
0808 02:22:24.592355 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:24.592358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:24.592414 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:24.592493 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:24.593709 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.594258 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.595728 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.596022 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.596166 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:24.597487 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:24.597490 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:24.597492 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:24.597510 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:24.597533 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:24.597542 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:24.597613 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:24.597808 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 02:22:24.597882 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 02:22:24.599113 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:24.589358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:41099 (local address 127.24.79.195:43485)
0808 02:22:24.590287 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:24.590368 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:24.592029 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:24.592794 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:24.592803 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:24.592909 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:24.593528 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:24.593546 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.594420 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.594423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:24.595553 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:24.595567 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.596219 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.596223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:24.596403 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:24.597128 (+ 725us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:22:24.597129 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:24.597182 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:22:24.597270 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:24.597648 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:24.597657 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:24.597663 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 02:22:24.597784 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 02:22:24.598083 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":199,"threads_started":1}
I20260808 02:22:24.600783 20267 heartbeater.cc:347] Connected to a master server at 127.24.79.254:41099
I20260808 02:22:24.601096 20267 heartbeater.cc:464] Registering TS with master...
I20260808 02:22:24.601820 20267 heartbeater.cc:511] Master 127.24.79.254:41099 requested a full tablet report, sending...
I20260808 02:22:24.603142 19562 ts_manager.cc:194] Registered new tserver with Master: d8905291085d4821b88165085e9ce4f7 (127.24.79.195:45887)
I20260808 02:22:24.603881 19562 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.79.195:43485
I20260808 02:22:24.603960 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:22:24.611574 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:24.606395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37994 (local address 127.24.79.254:41099)
0808 02:22:24.606549 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:24.606555 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:24.606895 (+ 340us) server_negotiation.cc:407] Connection header received
0808 02:22:24.607050 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:24.607053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:24.607115 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:24.607210 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:24.607682 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.608266 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.609221 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.609574 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.609685 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:24.610782 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:24.610785 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:24.610788 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:24.610806 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:24.610830 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:24.610839 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:24.611150 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:24.611348 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 02:22:24.611405 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:22:24.615764 19562 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37994:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:22:24.619064 19562 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:22:24.636408 20289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:24.631191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59648 (local address 127.24.79.193:37507)
0808 02:22:24.631741 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:24.631749 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:24.631776 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:22:24.631850 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:24.631855 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:24.632048 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:24.632500 (+ 452us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:24.632646 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.633663 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.634834 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.635608 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.635733 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:24.635977 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:24.636115 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 02:22:24.636235 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":97,"threads_started":1}
I20260808 02:22:24.636408 20288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:24.630750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:37507 (local address 127.0.0.1:59648)
0808 02:22:24.631408 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:24.631422 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:24.631506 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:24.632215 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:24.632218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:24.632253 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:24.632345 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:24.632351 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.633777 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.633780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:24.634696 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:24.634707 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.635767 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.635771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:24.635890 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:24.635931 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 02:22:24.636057 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 02:22:24.636228 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":2,"spinlock_wait_cycles":4864,"thread_start_us":92,"threads_started":1}
I20260808 02:22:24.636622 20284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:24.629925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:45887 (local address 127.0.0.1:45870)
0808 02:22:24.630893 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:24.630986 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:24.631154 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:24.631804 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:24.631811 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:24.631837 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:24.632000 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:24.632007 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.633458 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.633462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:24.634870 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:24.634880 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.636004 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.636008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:24.636271 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:24.636295 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 02:22:24.636348 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 02:22:24.636426 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":771,"mutex_wait_us":24,"spinlock_wait_cycles":6272,"thread_start_us":95,"threads_started":1}
I20260808 02:22:24.636893 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:24.630303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:37061 (local address 127.0.0.1:36022)
0808 02:22:24.630897 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:24.630986 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:24.631154 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:24.632216 (+ 1062us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:24.632219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:24.632246 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:24.632344 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:24.632349 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.633377 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.633380 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:24.635205 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:24.635214 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.636509 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.636513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:24.636612 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:24.636635 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:22:24.636702 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 02:22:24.636774 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"mutex_wait_us":415,"spinlock_wait_cycles":8960,"thread_start_us":124,"threads_started":1}
I20260808 02:22:24.637409 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:24.630090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45870 (local address 127.24.79.195:45887)
0808 02:22:24.631211 (+ 1121us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:24.631222 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:24.631251 (+ 29us) server_negotiation.cc:407] Connection header received
0808 02:22:24.631338 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:24.631344 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:24.631576 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:24.631790 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:24.632154 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.633345 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.635061 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.635853 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.636774 (+ 921us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:24.636979 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:24.637111 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 02:22:24.637234 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1010,"thread_start_us":206,"threads_started":1}
I20260808 02:22:24.637429 20287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:24.630473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36022 (local address 127.24.79.194:37061)
0808 02:22:24.630969 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:24.630983 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:24.631302 (+ 319us) server_negotiation.cc:407] Connection header received
0808 02:22:24.631372 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:24.631376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:24.631576 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:24.632042 (+ 466us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:24.632467 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.633250 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.635345 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.636149 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.636251 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:24.636448 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:24.637181 (+ 733us) server_negotiation.cc:299] Negotiation successful
0808 02:22:24.637277 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":136,"threads_started":1}
I20260808 02:22:24.640460 20154 tablet_service.cc:1467] Processing CreateTablet for tablet 558614dc45274c0297058b0530556bc7 (DEFAULT_TABLE table=test-table [id=a5f0d494143a4df98326f9c900abbaab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:22:24.640352 19960 tablet_service.cc:1467] Processing CreateTablet for tablet 558614dc45274c0297058b0530556bc7 (DEFAULT_TABLE table=test-table [id=a5f0d494143a4df98326f9c900abbaab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:22:24.640352 19763 tablet_service.cc:1467] Processing CreateTablet for tablet 558614dc45274c0297058b0530556bc7 (DEFAULT_TABLE table=test-table [id=a5f0d494143a4df98326f9c900abbaab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:22:24.641582 20154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 558614dc45274c0297058b0530556bc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:24.641597 19763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 558614dc45274c0297058b0530556bc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:24.641597 19960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 558614dc45274c0297058b0530556bc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:24.649683 20290 tablet_bootstrap.cc:492] T 558614dc45274c0297058b0530556bc7 P d8905291085d4821b88165085e9ce4f7: Bootstrap starting.
I20260808 02:22:24.651347 20292 tablet_bootstrap.cc:492] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322: Bootstrap starting.
I20260808 02:22:24.653172 20290 tablet_bootstrap.cc:654] T 558614dc45274c0297058b0530556bc7 P d8905291085d4821b88165085e9ce4f7: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:24.653651 20292 tablet_bootstrap.cc:654] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:24.654078 20290 log.cc:824] T 558614dc45274c0297058b0530556bc7 P d8905291085d4821b88165085e9ce4f7: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:24.654407 20292 log.cc:824] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:24.655049 20291 tablet_bootstrap.cc:492] T 558614dc45274c0297058b0530556bc7 P 97ed2c344a324a67958fa48707a324d6: Bootstrap starting.
I20260808 02:22:24.656701 20290 tablet_bootstrap.cc:492] T 558614dc45274c0297058b0530556bc7 P d8905291085d4821b88165085e9ce4f7: No bootstrap required, opened a new log
I20260808 02:22:24.656970 20290 ts_tablet_manager.cc:1397] T 558614dc45274c0297058b0530556bc7 P d8905291085d4821b88165085e9ce4f7: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 02:22:24.657238 20291 tablet_bootstrap.cc:654] T 558614dc45274c0297058b0530556bc7 P 97ed2c344a324a67958fa48707a324d6: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:24.657284 20292 tablet_bootstrap.cc:492] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322: No bootstrap required, opened a new log
I20260808 02:22:24.657514 20292 ts_tablet_manager.cc:1397] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 02:22:24.658082 20291 log.cc:824] T 558614dc45274c0297058b0530556bc7 P 97ed2c344a324a67958fa48707a324d6: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:24.660540 20291 tablet_bootstrap.cc:492] T 558614dc45274c0297058b0530556bc7 P 97ed2c344a324a67958fa48707a324d6: No bootstrap required, opened a new log
I20260808 02:22:24.660791 20291 ts_tablet_manager.cc:1397] T 558614dc45274c0297058b0530556bc7 P 97ed2c344a324a67958fa48707a324d6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 02:22:24.661834 20290 raft_consensus.cc:348] T 558614dc45274c0297058b0530556bc7 P d8905291085d4821b88165085e9ce4f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8905291085d4821b88165085e9ce4f7" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 45887 } } peers { permanent_uuid: "97ed2c344a324a67958fa48707a324d6" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 37507 } } peers { permanent_uuid: "4eabec7a8a2743928f4a89c5afbc7322" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37061 } }
I20260808 02:22:24.661834 20292 raft_consensus.cc:348] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8905291085d4821b88165085e9ce4f7" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 45887 } } peers { permanent_uuid: "97ed2c344a324a67958fa48707a324d6" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 37507 } } peers { permanent_uuid: "4eabec7a8a2743928f4a89c5afbc7322" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37061 } }
I20260808 02:22:24.662169 20290 raft_consensus.cc:374] T 558614dc45274c0297058b0530556bc7 P d8905291085d4821b88165085e9ce4f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:24.662170 20292 raft_consensus.cc:374] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:24.662235 20292 raft_consensus.cc:729] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eabec7a8a2743928f4a89c5afbc7322, State: Initialized, Role: FOLLOWER
I20260808 02:22:24.662235 20290 raft_consensus.cc:729] T 558614dc45274c0297058b0530556bc7 P d8905291085d4821b88165085e9ce4f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8905291085d4821b88165085e9ce4f7, State: Initialized, Role: FOLLOWER
I20260808 02:22:24.662796 20290 consensus_queue.cc:260] T 558614dc45274c0297058b0530556bc7 P d8905291085d4821b88165085e9ce4f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8905291085d4821b88165085e9ce4f7" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 45887 } } peers { permanent_uuid: "97ed2c344a324a67958fa48707a324d6" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 37507 } } peers { permanent_uuid: "4eabec7a8a2743928f4a89c5afbc7322" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37061 } }
I20260808 02:22:24.662796 20292 consensus_queue.cc:260] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8905291085d4821b88165085e9ce4f7" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 45887 } } peers { permanent_uuid: "97ed2c344a324a67958fa48707a324d6" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 37507 } } peers { permanent_uuid: "4eabec7a8a2743928f4a89c5afbc7322" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37061 } }
I20260808 02:22:24.663877 20061 heartbeater.cc:503] Master 127.24.79.254:41099 was elected leader, sending a full tablet report...
I20260808 02:22:24.663944 20267 heartbeater.cc:503] Master 127.24.79.254:41099 was elected leader, sending a full tablet report...
I20260808 02:22:24.664049 20292 ts_tablet_manager.cc:1428] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260808 02:22:24.664049 20290 ts_tablet_manager.cc:1428] T 558614dc45274c0297058b0530556bc7 P d8905291085d4821b88165085e9ce4f7: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260808 02:22:24.666471 20291 raft_consensus.cc:348] T 558614dc45274c0297058b0530556bc7 P 97ed2c344a324a67958fa48707a324d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8905291085d4821b88165085e9ce4f7" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 45887 } } peers { permanent_uuid: "97ed2c344a324a67958fa48707a324d6" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 37507 } } peers { permanent_uuid: "4eabec7a8a2743928f4a89c5afbc7322" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37061 } }
I20260808 02:22:24.666842 20291 raft_consensus.cc:374] T 558614dc45274c0297058b0530556bc7 P 97ed2c344a324a67958fa48707a324d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:24.666937 20291 raft_consensus.cc:729] T 558614dc45274c0297058b0530556bc7 P 97ed2c344a324a67958fa48707a324d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97ed2c344a324a67958fa48707a324d6, State: Initialized, Role: FOLLOWER
I20260808 02:22:24.667476 20291 consensus_queue.cc:260] T 558614dc45274c0297058b0530556bc7 P 97ed2c344a324a67958fa48707a324d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8905291085d4821b88165085e9ce4f7" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 45887 } } peers { permanent_uuid: "97ed2c344a324a67958fa48707a324d6" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 37507 } } peers { permanent_uuid: "4eabec7a8a2743928f4a89c5afbc7322" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37061 } }
I20260808 02:22:24.668460 19869 heartbeater.cc:503] Master 127.24.79.254:41099 was elected leader, sending a full tablet report...
I20260808 02:22:24.669574 20291 ts_tablet_manager.cc:1428] T 558614dc45274c0297058b0530556bc7 P 97ed2c344a324a67958fa48707a324d6: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260808 02:22:24.757544 20299 raft_consensus.cc:482] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:22:24.757877 20299 raft_consensus.cc:504] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8905291085d4821b88165085e9ce4f7" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 45887 } } peers { permanent_uuid: "97ed2c344a324a67958fa48707a324d6" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 37507 } } peers { permanent_uuid: "4eabec7a8a2743928f4a89c5afbc7322" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37061 } }
I20260808 02:22:24.759261 20299 leader_election.cc:288] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8905291085d4821b88165085e9ce4f7 (127.24.79.195:45887), 97ed2c344a324a67958fa48707a324d6 (127.24.79.193:37507)
I20260808 02:22:24.763873 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:24.759780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:48469 (local address 127.24.79.195:45887)
0808 02:22:24.759917 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:24.759922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:24.759939 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:22:24.760039 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:24.760042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:24.760101 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:24.760186 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:24.760597 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.761263 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.762419 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.763133 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.763252 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:24.763300 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:24.763669 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 02:22:24.763761 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 02:22:24.763862 20063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:24.759622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:45887 (local address 127.24.79.194:48469)
0808 02:22:24.759800 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:24.759816 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:24.759938 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:24.760326 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:24.760329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:24.760352 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:24.760468 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:24.760474 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.761398 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.761401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:24.762267 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:24.762278 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.763287 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.763291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:24.763404 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:24.763435 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 02:22:24.763533 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 02:22:24.763682 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 02:22:24.764724 20303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:24.759976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:37507 (local address 127.24.79.194:32991)
0808 02:22:24.760457 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:24.760471 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:24.760658 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:24.760978 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:24.760981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:24.761001 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:24.761149 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:24.761157 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.761965 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.761969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:24.763069 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:24.763082 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.764287 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.764291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:24.764389 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:24.764416 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 02:22:24.764501 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 02:22:24.764577 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":148,"threads_started":1}
I20260808 02:22:24.764778 20289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:24.760162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:32991 (local address 127.24.79.193:37507)
0808 02:22:24.760298 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:24.760302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:24.760559 (+ 257us) server_negotiation.cc:407] Connection header received
0808 02:22:24.760796 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:24.760801 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:24.760860 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:24.760940 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:24.761272 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.761842 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.763252 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.763912 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.764064 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:24.764174 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:24.764574 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 02:22:24.764673 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:22:24.764772 20183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "558614dc45274c0297058b0530556bc7" candidate_uuid: "4eabec7a8a2743928f4a89c5afbc7322" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8905291085d4821b88165085e9ce4f7" is_pre_election: true
I20260808 02:22:24.765275 20183 raft_consensus.cc:2454] T 558614dc45274c0297058b0530556bc7 P d8905291085d4821b88165085e9ce4f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4eabec7a8a2743928f4a89c5afbc7322 in term 0.
I20260808 02:22:24.765470 19797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "558614dc45274c0297058b0530556bc7" candidate_uuid: "4eabec7a8a2743928f4a89c5afbc7322" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97ed2c344a324a67958fa48707a324d6" is_pre_election: true
I20260808 02:22:24.765856 19797 raft_consensus.cc:2454] T 558614dc45274c0297058b0530556bc7 P 97ed2c344a324a67958fa48707a324d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4eabec7a8a2743928f4a89c5afbc7322 in term 0.
I20260808 02:22:24.765893 19894 leader_election.cc:302] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4eabec7a8a2743928f4a89c5afbc7322, d8905291085d4821b88165085e9ce4f7; no voters:
I20260808 02:22:24.766240 20299 raft_consensus.cc:2781] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:22:24.766355 20299 raft_consensus.cc:482] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:22:24.766412 20299 raft_consensus.cc:3037] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:24.767634 20299 raft_consensus.cc:504] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8905291085d4821b88165085e9ce4f7" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 45887 } } peers { permanent_uuid: "97ed2c344a324a67958fa48707a324d6" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 37507 } } peers { permanent_uuid: "4eabec7a8a2743928f4a89c5afbc7322" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37061 } }
I20260808 02:22:24.768002 20299 leader_election.cc:288] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322 [CANDIDATE]: Term 1 election: Requested vote from peers d8905291085d4821b88165085e9ce4f7 (127.24.79.195:45887), 97ed2c344a324a67958fa48707a324d6 (127.24.79.193:37507)
I20260808 02:22:24.768477 20183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "558614dc45274c0297058b0530556bc7" candidate_uuid: "4eabec7a8a2743928f4a89c5afbc7322" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8905291085d4821b88165085e9ce4f7"
I20260808 02:22:24.768628 20183 raft_consensus.cc:3037] T 558614dc45274c0297058b0530556bc7 P d8905291085d4821b88165085e9ce4f7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:24.768596 19797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "558614dc45274c0297058b0530556bc7" candidate_uuid: "4eabec7a8a2743928f4a89c5afbc7322" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97ed2c344a324a67958fa48707a324d6"
I20260808 02:22:24.768795 19797 raft_consensus.cc:3037] T 558614dc45274c0297058b0530556bc7 P 97ed2c344a324a67958fa48707a324d6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:24.769966 20183 raft_consensus.cc:2454] T 558614dc45274c0297058b0530556bc7 P d8905291085d4821b88165085e9ce4f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4eabec7a8a2743928f4a89c5afbc7322 in term 1.
I20260808 02:22:24.770031 19797 raft_consensus.cc:2454] T 558614dc45274c0297058b0530556bc7 P 97ed2c344a324a67958fa48707a324d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4eabec7a8a2743928f4a89c5afbc7322 in term 1.
I20260808 02:22:24.770437 19894 leader_election.cc:302] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4eabec7a8a2743928f4a89c5afbc7322, 97ed2c344a324a67958fa48707a324d6; no voters:
I20260808 02:22:24.770787 20299 raft_consensus.cc:2781] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:24.771126 20299 raft_consensus.cc:686] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322 [term 1 LEADER]: Becoming Leader. State: Replica: 4eabec7a8a2743928f4a89c5afbc7322, State: Running, Role: LEADER
I20260808 02:22:24.771641 20299 consensus_queue.cc:237] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8905291085d4821b88165085e9ce4f7" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 45887 } } peers { permanent_uuid: "97ed2c344a324a67958fa48707a324d6" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 37507 } } peers { permanent_uuid: "4eabec7a8a2743928f4a89c5afbc7322" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37061 } }
I20260808 02:22:24.775452 19571 catalog_manager.cc:5495] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4eabec7a8a2743928f4a89c5afbc7322 (127.24.79.194). New cstate: current_term: 1 leader_uuid: "4eabec7a8a2743928f4a89c5afbc7322" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8905291085d4821b88165085e9ce4f7" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 45887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97ed2c344a324a67958fa48707a324d6" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 37507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4eabec7a8a2743928f4a89c5afbc7322" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 37061 } health_report { overall_health: HEALTHY } } }
I20260808 02:22:24.801810 20287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:24.797796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36032 (local address 127.24.79.194:37061)
0808 02:22:24.797958 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:24.797961 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:24.797974 (+ 13us) server_negotiation.cc:407] Connection header received
0808 02:22:24.798056 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:24.798058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:24.798142 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:24.798348 (+ 206us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:22:24.798637 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.799235 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.800140 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:24.800364 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:24.800457 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:24.800838 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:22:24.801318 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:22:24.801470 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:24.801601 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 02:22:24.801653 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 02:22:24.809137 20183 raft_consensus.cc:1261] T 558614dc45274c0297058b0530556bc7 P d8905291085d4821b88165085e9ce4f7 [term 1 FOLLOWER]: Refusing update from remote peer 4eabec7a8a2743928f4a89c5afbc7322: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:22:24.809178 19797 raft_consensus.cc:1261] T 558614dc45274c0297058b0530556bc7 P 97ed2c344a324a67958fa48707a324d6 [term 1 FOLLOWER]: Refusing update from remote peer 4eabec7a8a2743928f4a89c5afbc7322: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:22:24.810005 20299 consensus_queue.cc:1035] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8905291085d4821b88165085e9ce4f7" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 45887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:22:24.810319 20304 consensus_queue.cc:1035] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97ed2c344a324a67958fa48707a324d6" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 37507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:22:24.820945 20311 mvcc.cc:205] Tried to move back new op lower bound from 7316093930728722432 to 7316093930587684864. Current Snapshot: MvccSnapshot[applied={T|T < 7316093930728722432}]
I20260808 02:22:24.821171 20310 mvcc.cc:205] Tried to move back new op lower bound from 7316093930728722432 to 7316093930587684864. Current Snapshot: MvccSnapshot[applied={T|T < 7316093930728722432}]
I20260808 02:22:24.830181 19571 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:22:24.830368 19571 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:22:24.833621 19571 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a21f756013a4965aec5d56f3ffc580e: Sending DeleteTablet for 3 replicas of tablet 558614dc45274c0297058b0530556bc7
I20260808 02:22:24.834676 19960 tablet_service.cc:1514] Processing DeleteTablet for tablet 558614dc45274c0297058b0530556bc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:22:24 UTC) from {username='slave'} at 127.0.0.1:36022
I20260808 02:22:24.834893 20154 tablet_service.cc:1514] Processing DeleteTablet for tablet 558614dc45274c0297058b0530556bc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:22:24 UTC) from {username='slave'} at 127.0.0.1:45870
I20260808 02:22:24.835456 19763 tablet_service.cc:1514] Processing DeleteTablet for tablet 558614dc45274c0297058b0530556bc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:22:24 UTC) from {username='slave'} at 127.0.0.1:59648
I20260808 02:22:24.836256 20326 tablet_replica.cc:330] T 558614dc45274c0297058b0530556bc7 P d8905291085d4821b88165085e9ce4f7: stopping tablet replica
I20260808 02:22:24.836678 20326 raft_consensus.cc:2229] T 558614dc45274c0297058b0530556bc7 P d8905291085d4821b88165085e9ce4f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:22:24.836735 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 19664
I20260808 02:22:24.837118 20326 raft_consensus.cc:2258] T 558614dc45274c0297058b0530556bc7 P d8905291085d4821b88165085e9ce4f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:22:24.847887 20326 ts_tablet_manager.cc:1905] T 558614dc45274c0297058b0530556bc7 P d8905291085d4821b88165085e9ce4f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:22:24.849174 20325 tablet_replica.cc:330] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322: stopping tablet replica
I20260808 02:22:24.849671 20325 raft_consensus.cc:2229] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:22:24.851514 20326 ts_tablet_manager.cc:1918] T 558614dc45274c0297058b0530556bc7 P d8905291085d4821b88165085e9ce4f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:22:24.851665 20326 log.cc:1196] T 558614dc45274c0297058b0530556bc7 P d8905291085d4821b88165085e9ce4f7: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/558614dc45274c0297058b0530556bc7
I20260808 02:22:24.852042 20326 ts_tablet_manager.cc:1939] T 558614dc45274c0297058b0530556bc7 P d8905291085d4821b88165085e9ce4f7: Deleting consensus metadata
I20260808 02:22:24.852808 20325 raft_consensus.cc:2258] T 558614dc45274c0297058b0530556bc7 P 4eabec7a8a2743928f4a89c5afbc7322 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:22:24.853861 19548 catalog_manager.cc:4841] TS d8905291085d4821b88165085e9ce4f7 (127.24.79.195:45887): tablet 558614dc45274c0297058b0530556bc7 (table test-table [id=a5f0d494143a4df98326f9c900abbaab]) successfully deleted
W20260808 02:22:24.856313 19548 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.79.193:37507 (error 108)
I20260808 02:22:24.857098 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 19873
I20260808 02:22:24.881906 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:24.881484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:37507 (local address 127.0.0.1:59662)
0808 02:22:24.881680 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:24.881800 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.193:37507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260808 02:22:24.882193 19548 catalog_manager.cc:4584] TS 97ed2c344a324a67958fa48707a324d6 (127.24.79.193:37507): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 558614dc45274c0297058b0530556bc7: Network error: Client connection negotiation failed: client connection to 127.24.79.193:37507: connect: Connection refused (error 111)
I20260808 02:22:24.887805 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 20064
I20260808 02:22:24.890987 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:24.888055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:37061 (local address 127.0.0.1:36042)
0808 02:22:24.890737 (+ 2682us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:24.890831 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.194:37061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2533}
I20260808 02:22:24.900316 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 19529
2026-08-08T02:22:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1310 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:22:24 dist-test-slave-1vmg krb5kdc[20332](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:22:24 dist-test-slave-1vmg krb5kdc[20332](info): set up 2 sockets
Aug 08 02:22:24 dist-test-slave-1vmg krb5kdc[20332](info): commencing operation
krb5kdc: starting...
W20260808 02:22:26.091496 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:22:26 dist-test-slave-1vmg krb5kdc[20332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:22:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:22:26.252359 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:38879
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:36455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:38879
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:26.383533 20352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:26.383910 20352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:26.383978 20352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:26.388372 20352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:26.388477 20352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:26.388509 20352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:26.388532 20352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:26.388551 20352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:26.394662 20352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:36455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:38879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:38879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:26.396049 20352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:26.397277 20352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:26.407260 20360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:26.407269 20359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:26.407569 20352 server_base.cc:1034] running on GCE node
W20260808 02:22:26.407262 20362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:26.408340 20352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:26.409718 20352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:26.410969 20352 hybrid_clock.cc:648] HybridClock initialized: now 1786155746410910 us; error 78 us; skew 500 ppm
Aug 08 02:22:26 dist-test-slave-1vmg krb5kdc[20332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155746, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:26.414549 20352 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:22:26.415875 20352 webserver.cc:460] Webserver started at http://127.24.79.254:35453/ using document root <none> and password file <none>
I20260808 02:22:26.416568 20352 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:26.416657 20352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:26.416921 20352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:26.419428 20352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "6977df97d93b4cf9a812129dda2072c5"
format_stamp: "Formatted at 2026-08-08 02:22:26 on dist-test-slave-1vmg"
server_key: "f913046d8509d00511f4736f266464ed"
server_key_iv: "a364ef6a461bccbd127f3e4110621a41"
server_key_version: "encryptionkey@0"
I20260808 02:22:26.420168 20352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "6977df97d93b4cf9a812129dda2072c5"
format_stamp: "Formatted at 2026-08-08 02:22:26 on dist-test-slave-1vmg"
server_key: "f913046d8509d00511f4736f266464ed"
server_key_iv: "a364ef6a461bccbd127f3e4110621a41"
server_key_version: "encryptionkey@0"
I20260808 02:22:26.424816 20352 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:22:26.427935 20370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:26.429589 20352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:22:26.429774 20352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "6977df97d93b4cf9a812129dda2072c5"
format_stamp: "Formatted at 2026-08-08 02:22:26 on dist-test-slave-1vmg"
server_key: "f913046d8509d00511f4736f266464ed"
server_key_iv: "a364ef6a461bccbd127f3e4110621a41"
server_key_version: "encryptionkey@0"
I20260808 02:22:26.429919 20352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:26.448876 20352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:26.453182 20352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:26.453421 20352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:26.462070 20352 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:38879
I20260808 02:22:26.462097 20449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:38879 every 8 connection(s)
I20260808 02:22:26.463295 20352 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:26.466717 20450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:26.472191 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 20352
I20260808 02:22:26.472344 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:26.472631 24895 external_mini_cluster.cc:1428] Setting key d3392e47af23fa2f3bde59450c4e4ec7
I20260808 02:22:26.474062 20450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6977df97d93b4cf9a812129dda2072c5: Bootstrap starting.
I20260808 02:22:26.476866 20450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6977df97d93b4cf9a812129dda2072c5: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:26.478186 20450 log.cc:824] T 00000000000000000000000000000000 P 6977df97d93b4cf9a812129dda2072c5: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:26.480986 20450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6977df97d93b4cf9a812129dda2072c5: No bootstrap required, opened a new log
Aug 08 02:22:26 dist-test-slave-1vmg krb5kdc[20332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:22:26.485445 20450 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6977df97d93b4cf9a812129dda2072c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6977df97d93b4cf9a812129dda2072c5" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38879 } }
I20260808 02:22:26.485827 20450 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6977df97d93b4cf9a812129dda2072c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:26.485877 20450 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6977df97d93b4cf9a812129dda2072c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6977df97d93b4cf9a812129dda2072c5, State: Initialized, Role: FOLLOWER
I20260808 02:22:26.486413 20450 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6977df97d93b4cf9a812129dda2072c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6977df97d93b4cf9a812129dda2072c5" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38879 } }
I20260808 02:22:26.486564 20450 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6977df97d93b4cf9a812129dda2072c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:26.486635 20450 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6977df97d93b4cf9a812129dda2072c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:26.486745 20450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6977df97d93b4cf9a812129dda2072c5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:26.488157 20450 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6977df97d93b4cf9a812129dda2072c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6977df97d93b4cf9a812129dda2072c5" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38879 } }
I20260808 02:22:26.488620 20450 leader_election.cc:302] T 00000000000000000000000000000000 P 6977df97d93b4cf9a812129dda2072c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6977df97d93b4cf9a812129dda2072c5; no voters:
I20260808 02:22:26.488996 20450 leader_election.cc:288] T 00000000000000000000000000000000 P 6977df97d93b4cf9a812129dda2072c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:22:26.489122 20456 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6977df97d93b4cf9a812129dda2072c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:26.489290 20456 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6977df97d93b4cf9a812129dda2072c5 [term 1 LEADER]: Becoming Leader. State: Replica: 6977df97d93b4cf9a812129dda2072c5, State: Running, Role: LEADER
I20260808 02:22:26.489643 20456 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6977df97d93b4cf9a812129dda2072c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6977df97d93b4cf9a812129dda2072c5" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38879 } }
I20260808 02:22:26.490518 20450 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6977df97d93b4cf9a812129dda2072c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:26.491112 20456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6977df97d93b4cf9a812129dda2072c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6977df97d93b4cf9a812129dda2072c5. Latest consensus state: current_term: 1 leader_uuid: "6977df97d93b4cf9a812129dda2072c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6977df97d93b4cf9a812129dda2072c5" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38879 } } }
I20260808 02:22:26.491252 20453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:26.474444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40972 (local address 127.24.79.254:38879)
0808 02:22:26.475009 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:26.475030 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:26.475074 (+ 44us) server_negotiation.cc:407] Connection header received
0808 02:22:26.475854 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:26.475876 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:26.476289 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:26.476641 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:26.477953 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:26.479182 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:26.480489 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:26.481071 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:26.481276 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:26.484296 (+ 3020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:26.484303 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:26.484324 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:26.484366 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:26.486876 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:26.487606 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:26.487612 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:26.487618 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:26.487773 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:26.488183 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:26.488187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:26.488188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:26.488962 (+ 774us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:26.489189 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:26.489702 (+ 513us) server_negotiation.cc:299] Negotiation successful
0808 02:22:26.490086 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":169,"threads_started":1}
I20260808 02:22:26.491410 20458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6977df97d93b4cf9a812129dda2072c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6977df97d93b4cf9a812129dda2072c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6977df97d93b4cf9a812129dda2072c5" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38879 } } }
I20260808 02:22:26.491514 20458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6977df97d93b4cf9a812129dda2072c5 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:26.491254 20456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6977df97d93b4cf9a812129dda2072c5 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:26.492209 20466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:22:26.496169 20466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:22:26.503283 20466 catalog_manager.cc:1347] Generated new cluster ID: e911bf515e2641129bc7e7ed0bd3f2e5
I20260808 02:22:26.503407 20466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:22:26.511602 20466 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:22:26.513087 20466 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:22:26.524588 20466 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6977df97d93b4cf9a812129dda2072c5: Generated new TSK 0
I20260808 02:22:26.525489 20466 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.79.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
Entry for principal kudu/127.24.79.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab.
I20260808 02:22:26.572314 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:38879
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:36455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:26.726969 20485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:26.727375 20485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:26.727427 20485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:26.731740 20485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:26.731835 20485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:26.731899 20485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:22:26.737782 20485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:36455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.193.keytab
--principal=kudu/127.24.79.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:38879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:26.739099 20485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:26.740274 20485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:26.751421 20495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:26.751421 20499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:26.751523 20491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:26.752055 20485 server_base.cc:1034] running on GCE node
I20260808 02:22:26.752575 20485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:26.753330 20485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:26.754546 20485 hybrid_clock.cc:648] HybridClock initialized: now 1786155746754522 us; error 39 us; skew 500 ppm
Aug 08 02:22:26 dist-test-slave-1vmg krb5kdc[20332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155746, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:26.758163 20485 init.cc:377] Logged in from keytab as kudu/127.24.79.193@KRBTEST.COM (short username kudu)
I20260808 02:22:26.759424 20485 webserver.cc:460] Webserver started at http://127.24.79.193:45651/ using document root <none> and password file <none>
I20260808 02:22:26.760072 20485 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:26.760149 20485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:26.760388 20485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:26.762732 20485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "99538ddf05824967b355a8b16c832793"
format_stamp: "Formatted at 2026-08-08 02:22:26 on dist-test-slave-1vmg"
server_key: "44b18c9419958824f67ce1321ed270db"
server_key_iv: "f340437c68ee7ca38bc2781236e374fa"
server_key_version: "encryptionkey@0"
I20260808 02:22:26.763397 20485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "99538ddf05824967b355a8b16c832793"
format_stamp: "Formatted at 2026-08-08 02:22:26 on dist-test-slave-1vmg"
server_key: "44b18c9419958824f67ce1321ed270db"
server_key_iv: "f340437c68ee7ca38bc2781236e374fa"
server_key_version: "encryptionkey@0"
I20260808 02:22:26.767671 20485 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:22:26.770654 20510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:26.772145 20485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 02:22:26.772307 20485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "99538ddf05824967b355a8b16c832793"
format_stamp: "Formatted at 2026-08-08 02:22:26 on dist-test-slave-1vmg"
server_key: "44b18c9419958824f67ce1321ed270db"
server_key_iv: "f340437c68ee7ca38bc2781236e374fa"
server_key_version: "encryptionkey@0"
I20260808 02:22:26.772447 20485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:26.784612 20485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:26.788558 20485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:26.788777 20485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:26.789469 20485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:22:26.790423 20485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:22:26.790477 20485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:26.790534 20485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:22:26.790566 20485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:26.800247 20485 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:36041
I20260808 02:22:26.800331 20674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:36041 every 8 connection(s)
I20260808 02:22:26.801329 20485 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:22:26.803189 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 20485
I20260808 02:22:26.803369 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:22:26.803690 24895 external_mini_cluster.cc:1428] Setting key 6e9ba6be33bfa20edc56cb1834f85af1
Aug 08 02:22:26 dist-test-slave-1vmg krb5kdc[20332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155746, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.193@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.79.194@KRBTEST.COM; defaulting to no policy
I20260808 02:22:26.819147 20453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:26.805185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:47533 (local address 127.24.79.254:38879)
0808 02:22:26.805326 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:26.805330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:26.805345 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:22:26.806560 (+ 1215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:26.806563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:26.806614 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:26.806686 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:26.808452 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:26.809299 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:26.810850 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:26.811271 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:26.811400 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:26.814392 (+ 2992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:26.814398 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:26.814401 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:26.814441 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:26.816448 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:26.817174 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:26.817177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:26.817179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:26.817280 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:26.817696 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:26.817699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:26.817700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:26.817943 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:26.818049 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:26.818751 (+ 702us) server_negotiation.cc:299] Negotiation successful
0808 02:22:26.818954 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.194@KRBTEST.COM" created.
I20260808 02:22:26.820426 20677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:26.804001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38879 (local address 127.24.79.193:47533)
0808 02:22:26.804633 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:26.804688 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:26.806281 (+ 1593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:26.806965 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:26.806976 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:26.807520 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:26.808243 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:26.808262 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:26.809478 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:26.809482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:26.810679 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:26.810693 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:26.811481 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:26.811485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:26.811688 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:26.812366 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:22:26.812386 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:26.814120 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:26.816651 (+ 2531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:26.816660 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:26.816680 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:26.817035 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:26.817388 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:26.817392 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:26.817394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:26.817578 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:26.818063 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:26.818072 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:26.818460 (+ 388us) client_negotiation.cc:769] Sending connection context
0808 02:22:26.818679 (+ 219us) client_negotiation.cc:240] Negotiation successful
0808 02:22:26.819227 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":162,"threads_started":1}
I20260808 02:22:26.822305 20675 heartbeater.cc:347] Connected to a master server at 127.24.79.254:38879
I20260808 02:22:26.822606 20675 heartbeater.cc:464] Registering TS with master...
I20260808 02:22:26.823315 20675 heartbeater.cc:511] Master 127.24.79.254:38879 requested a full tablet report, sending...
I20260808 02:22:26.825263 20395 ts_manager.cc:194] Registered new tserver with Master: 99538ddf05824967b355a8b16c832793 (127.24.79.193:36041)
I20260808 02:22:26.826521 20395 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.193@KRBTEST.COM'} at 127.24.79.193:47533
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
Entry for principal kudu/127.24.79.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab.
I20260808 02:22:26.839547 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:38879
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:36455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:26.972896 20680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:26.973338 20680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:26.973502 20680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:26.978394 20680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:26.978483 20680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:26.978562 20680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:22:26.984763 20680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:36455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.194.keytab
--principal=kudu/127.24.79.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:38879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:26.986025 20680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:26.987232 20680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:26.998286 20691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:26.998492 20680 server_base.cc:1034] running on GCE node
W20260808 02:22:26.998301 20690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:26.998301 20694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:26.999141 20680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:26.999953 20680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:27.001204 20680 hybrid_clock.cc:648] HybridClock initialized: now 1786155747001182 us; error 42 us; skew 500 ppm
Aug 08 02:22:27 dist-test-slave-1vmg krb5kdc[20332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155747, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:27.004711 20680 init.cc:377] Logged in from keytab as kudu/127.24.79.194@KRBTEST.COM (short username kudu)
I20260808 02:22:27.005862 20680 webserver.cc:460] Webserver started at http://127.24.79.194:34139/ using document root <none> and password file <none>
I20260808 02:22:27.006513 20680 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:27.006592 20680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:27.006831 20680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:27.009313 20680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "17e00872e0e549a68917d157bad38c21"
format_stamp: "Formatted at 2026-08-08 02:22:27 on dist-test-slave-1vmg"
server_key: "06fc64baaf4de129b717f44dcd52f655"
server_key_iv: "190ced9024babe60d08431009fd3ae12"
server_key_version: "encryptionkey@0"
I20260808 02:22:27.010260 20680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "17e00872e0e549a68917d157bad38c21"
format_stamp: "Formatted at 2026-08-08 02:22:27 on dist-test-slave-1vmg"
server_key: "06fc64baaf4de129b717f44dcd52f655"
server_key_iv: "190ced9024babe60d08431009fd3ae12"
server_key_version: "encryptionkey@0"
I20260808 02:22:27.015504 20680 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 02:22:27.018738 20706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:27.020653 20680 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:22:27.020833 20680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "17e00872e0e549a68917d157bad38c21"
format_stamp: "Formatted at 2026-08-08 02:22:27 on dist-test-slave-1vmg"
server_key: "06fc64baaf4de129b717f44dcd52f655"
server_key_iv: "190ced9024babe60d08431009fd3ae12"
server_key_version: "encryptionkey@0"
I20260808 02:22:27.020980 20680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:27.063359 20680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:27.067647 20680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:27.067893 20680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:27.068608 20680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:22:27.069589 20680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:22:27.069643 20680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:27.069701 20680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:22:27.069730 20680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:27.080479 20680 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:45841
I20260808 02:22:27.080556 20871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:45841 every 8 connection(s)
I20260808 02:22:27.081620 20680 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:22:27.089942 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 20680
I20260808 02:22:27.090134 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:22:27.090456 24895 external_mini_cluster.cc:1428] Setting key 2cd64e908567cb039d3dde67e778dc7f
Aug 08 02:22:27 dist-test-slave-1vmg krb5kdc[20332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155747, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.194@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:22:27.098623 20453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:27.083920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:46201 (local address 127.24.79.254:38879)
0808 02:22:27.084074 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:27.084078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:27.084997 (+ 919us) server_negotiation.cc:407] Connection header received
0808 02:22:27.086450 (+ 1453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:27.086453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:27.086505 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:27.086587 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:27.088095 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.088685 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.090082 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.090494 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.090669 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:27.093783 (+ 3114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:27.093787 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:27.093789 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:27.093820 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:27.095779 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:27.096556 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:27.096560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:27.096561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:27.096655 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:27.097133 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:27.097136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:27.097138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:27.097376 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:27.097467 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:27.098305 (+ 838us) server_negotiation.cc:299] Negotiation successful
0808 02:22:27.098477 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 02:22:27.100026 20876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:27.084262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38879 (local address 127.24.79.194:46201)
0808 02:22:27.084808 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:27.084859 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:27.086193 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:27.086780 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:27.086789 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:27.087272 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:27.087840 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:27.087866 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.088851 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.088854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:27.089893 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:27.089906 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.090700 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.090705 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:27.090970 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:27.091721 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:22:27.091741 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:27.093562 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:27.095958 (+ 2396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:27.095965 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:27.095984 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:27.096419 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:27.096764 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:27.096768 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:27.096770 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:27.096992 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:27.097498 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:27.097506 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:27.097940 (+ 434us) client_negotiation.cc:769] Sending connection context
0808 02:22:27.098216 (+ 276us) client_negotiation.cc:240] Negotiation successful
0808 02:22:27.098677 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":131,"threads_started":1}
I20260808 02:22:27.101763 20872 heartbeater.cc:347] Connected to a master server at 127.24.79.254:38879
I20260808 02:22:27.102059 20872 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.24.79.195@KRBTEST.COM; defaulting to no policy
I20260808 02:22:27.102667 20872 heartbeater.cc:511] Master 127.24.79.254:38879 requested a full tablet report, sending...
I20260808 02:22:27.103974 20395 ts_manager.cc:194] Registered new tserver with Master: 17e00872e0e549a68917d157bad38c21 (127.24.79.194:45841)
I20260808 02:22:27.104614 20395 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.194@KRBTEST.COM'} at 127.24.79.194:46201
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
Entry for principal kudu/127.24.79.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab.
I20260808 02:22:27.130949 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:38879
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:36455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:27.257054 20880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:27.257457 20880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:27.257508 20880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:27.261871 20880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:27.261976 20880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:27.262039 20880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:22:27.268322 20880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:36455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.195.keytab
--principal=kudu/127.24.79.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:38879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:27.269738 20880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:27.271229 20880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:27.283032 20886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:27.283032 20887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:27.283032 20893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:27.283612 20880 server_base.cc:1034] running on GCE node
I20260808 02:22:27.284406 20880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:27.285251 20880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:27.286532 20880 hybrid_clock.cc:648] HybridClock initialized: now 1786155747286482 us; error 70 us; skew 500 ppm
Aug 08 02:22:27 dist-test-slave-1vmg krb5kdc[20332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155747, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:27.294615 20880 init.cc:377] Logged in from keytab as kudu/127.24.79.195@KRBTEST.COM (short username kudu)
I20260808 02:22:27.295953 20880 webserver.cc:460] Webserver started at http://127.24.79.195:43787/ using document root <none> and password file <none>
I20260808 02:22:27.296672 20880 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:27.296764 20880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:27.297003 20880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:27.299724 20880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "85207e75b51e46cfaf374c194f3ad1b5"
format_stamp: "Formatted at 2026-08-08 02:22:27 on dist-test-slave-1vmg"
server_key: "dbdc9d2dd67bb95bdac1f093976af4bc"
server_key_iv: "49ca3d94e24c6dc4c8a037fa7a8743aa"
server_key_version: "encryptionkey@0"
I20260808 02:22:27.300540 20880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "85207e75b51e46cfaf374c194f3ad1b5"
format_stamp: "Formatted at 2026-08-08 02:22:27 on dist-test-slave-1vmg"
server_key: "dbdc9d2dd67bb95bdac1f093976af4bc"
server_key_iv: "49ca3d94e24c6dc4c8a037fa7a8743aa"
server_key_version: "encryptionkey@0"
I20260808 02:22:27.305646 20880 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260808 02:22:27.308948 20902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:27.310586 20880 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:22:27.310756 20880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "85207e75b51e46cfaf374c194f3ad1b5"
format_stamp: "Formatted at 2026-08-08 02:22:27 on dist-test-slave-1vmg"
server_key: "dbdc9d2dd67bb95bdac1f093976af4bc"
server_key_iv: "49ca3d94e24c6dc4c8a037fa7a8743aa"
server_key_version: "encryptionkey@0"
I20260808 02:22:27.310892 20880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:27.327294 20880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:27.332186 20880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:27.332448 20880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:27.333376 20880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:22:27.334856 20880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:22:27.334959 20880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260808 02:22:27.335040 20880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:22:27.335072 20880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:27.345953 20880 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:43579
I20260808 02:22:27.346020 21077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:43579 every 8 connection(s)
I20260808 02:22:27.347152 20880 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:22:27.350378 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 20880
I20260808 02:22:27.350539 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:22:27.350885 24895 external_mini_cluster.cc:1428] Setting key f1f6b707fc519371f0ebdab9bd40de96
Aug 08 02:22:27 dist-test-slave-1vmg krb5kdc[20332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155747, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.195@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:22:27.364711 20453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:27.349533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:46497 (local address 127.24.79.254:38879)
0808 02:22:27.349738 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:27.349742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:27.351007 (+ 1265us) server_negotiation.cc:407] Connection header received
0808 02:22:27.352938 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:27.352942 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:27.353005 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:27.353079 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:27.354650 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.355251 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.356777 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.357007 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.357120 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:27.360334 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:27.360338 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:27.360340 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:27.360376 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:27.362044 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:27.362758 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:27.362762 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:27.362764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:27.362861 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:27.363330 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:27.363334 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:27.363335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:27.363565 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:27.363675 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:27.364364 (+ 689us) server_negotiation.cc:299] Negotiation successful
0808 02:22:27.364523 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 02:22:27.365800 21080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:27.349928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38879 (local address 127.24.79.195:46497)
0808 02:22:27.350662 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:27.350804 (+ 142us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:27.352659 (+ 1855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:27.353312 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:27.353321 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:27.353828 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:27.354465 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:27.354483 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.355399 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.355402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:27.356593 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:27.356608 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.357201 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.357205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:27.357401 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:27.358117 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:22:27.358139 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:27.360053 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:27.362241 (+ 2188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:27.362249 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:27.362270 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:27.362623 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:27.362983 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:27.362986 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:27.362988 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:27.363194 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:27.363690 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:27.363700 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:27.364066 (+ 366us) client_negotiation.cc:769] Sending connection context
0808 02:22:27.364272 (+ 206us) client_negotiation.cc:240] Negotiation successful
0808 02:22:27.364686 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":240,"threads_started":1}
I20260808 02:22:27.367329 21078 heartbeater.cc:347] Connected to a master server at 127.24.79.254:38879
I20260808 02:22:27.367638 21078 heartbeater.cc:464] Registering TS with master...
I20260808 02:22:27.368306 21078 heartbeater.cc:511] Master 127.24.79.254:38879 requested a full tablet report, sending...
I20260808 02:22:27.369678 20395 ts_manager.cc:194] Registered new tserver with Master: 85207e75b51e46cfaf374c194f3ad1b5 (127.24.79.195:43579)
I20260808 02:22:27.370306 20395 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.79.195@KRBTEST.COM'} at 127.24.79.195:46497
I20260808 02:22:27.378994 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:22:27.389536 20453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:27.381297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40986 (local address 127.24.79.254:38879)
0808 02:22:27.381481 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:27.381485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:27.381586 (+ 101us) server_negotiation.cc:407] Connection header received
0808 02:22:27.381781 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:27.381783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:27.381836 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:27.381934 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:27.382741 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.383330 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.384183 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.384411 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.384509 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:27.385999 (+ 1490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:27.386003 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:27.386005 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:27.386036 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:27.387639 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:27.388144 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:27.388148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:27.388149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:27.388253 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:27.388535 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:27.388538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:27.388540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:27.388807 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:27.388943 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:27.389235 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 02:22:27.389365 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 02:22:27.393448 20395 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:22:27.396823 20395 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:22:27.413621 21102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:27.408558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37310 (local address 127.24.79.195:43579)
0808 02:22:27.409102 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:27.409114 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:27.409137 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:22:27.409216 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:27.409220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:27.409427 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:27.409588 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:27.409795 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.410775 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.411966 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.412862 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.412969 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:27.413149 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:27.413348 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 02:22:27.413458 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":199,"threads_started":1}
I20260808 02:22:27.413612 21101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:27.407557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:43579 (local address 127.0.0.1:37310)
0808 02:22:27.408101 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:27.408115 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:27.408217 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:27.409565 (+ 1348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:27.409568 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:27.409588 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:27.409697 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:27.409705 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.410905 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.410908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:27.411800 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:27.411814 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.412989 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.412993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:27.413124 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:27.413168 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 02:22:27.413291 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 02:22:27.413446 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":137,"threads_started":1}
I20260808 02:22:27.413985 21099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:27.407441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:36041 (local address 127.0.0.1:37566)
0808 02:22:27.407871 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:27.407930 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:27.408100 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:27.408823 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:27.408830 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:27.408860 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:27.409040 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:27.409051 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.410543 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.410547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:27.412036 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:27.412050 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.413499 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.413504 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:27.413621 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:27.413650 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 02:22:27.413731 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 02:22:27.413823 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":1,"thread_start_us":123,"threads_started":1}
I20260808 02:22:27.414675 21100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:27.407546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37566 (local address 127.24.79.193:36041)
0808 02:22:27.408309 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:27.408317 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:27.408340 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:22:27.408407 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:27.408411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:27.408640 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:27.408818 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:27.409215 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.410363 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.412207 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.413133 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.413260 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:27.413428 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:27.414304 (+ 876us) server_negotiation.cc:299] Negotiation successful
0808 02:22:27.414493 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":113,"threads_started":1}
I20260808 02:22:27.415685 21103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:27.409051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:45841 (local address 127.0.0.1:35930)
0808 02:22:27.409928 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:27.409943 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:27.410040 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:27.410500 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:27.410502 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:27.410521 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:27.410614 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:27.410619 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.412158 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.412161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:27.413304 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:27.413317 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.415166 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.415171 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:27.415336 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:27.415364 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 02:22:27.415446 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 02:22:27.415534 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":811,"thread_start_us":114,"threads_started":1}
I20260808 02:22:27.415795 21104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:27.409230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35930 (local address 127.24.79.194:45841)
0808 02:22:27.409897 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:27.409911 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:27.410040 (+ 129us) server_negotiation.cc:407] Connection header received
0808 02:22:27.410151 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:27.410156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:27.410347 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:27.410512 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:27.410697 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.411966 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.413475 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.414748 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.414879 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:27.415085 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:27.415530 (+ 445us) server_negotiation.cc:299] Negotiation successful
0808 02:22:27.415648 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":128,"threads_started":1}
I20260808 02:22:27.417467 20971 tablet_service.cc:1467] Processing CreateTablet for tablet 29e46c96eaf143549b618ead37b157b0 (DEFAULT_TABLE table=test-table [id=d34383ff1a174a1eb1713e91b0b9cc14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:22:27.418771 20971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29e46c96eaf143549b618ead37b157b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:27.418519 20581 tablet_service.cc:1467] Processing CreateTablet for tablet 29e46c96eaf143549b618ead37b157b0 (DEFAULT_TABLE table=test-table [id=d34383ff1a174a1eb1713e91b0b9cc14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:22:27.418649 20769 tablet_service.cc:1467] Processing CreateTablet for tablet 29e46c96eaf143549b618ead37b157b0 (DEFAULT_TABLE table=test-table [id=d34383ff1a174a1eb1713e91b0b9cc14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:22:27.419627 20581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29e46c96eaf143549b618ead37b157b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:27.419812 20769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29e46c96eaf143549b618ead37b157b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:27.426551 21107 tablet_bootstrap.cc:492] T 29e46c96eaf143549b618ead37b157b0 P 99538ddf05824967b355a8b16c832793: Bootstrap starting.
I20260808 02:22:27.427625 21109 tablet_bootstrap.cc:492] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5: Bootstrap starting.
I20260808 02:22:27.429666 21107 tablet_bootstrap.cc:654] T 29e46c96eaf143549b618ead37b157b0 P 99538ddf05824967b355a8b16c832793: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:27.430019 21109 tablet_bootstrap.cc:654] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:27.430045 21110 tablet_bootstrap.cc:492] T 29e46c96eaf143549b618ead37b157b0 P 17e00872e0e549a68917d157bad38c21: Bootstrap starting.
I20260808 02:22:27.430877 21107 log.cc:824] T 29e46c96eaf143549b618ead37b157b0 P 99538ddf05824967b355a8b16c832793: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:27.431246 21109 log.cc:824] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:27.432674 21110 tablet_bootstrap.cc:654] T 29e46c96eaf143549b618ead37b157b0 P 17e00872e0e549a68917d157bad38c21: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:27.433362 21107 tablet_bootstrap.cc:492] T 29e46c96eaf143549b618ead37b157b0 P 99538ddf05824967b355a8b16c832793: No bootstrap required, opened a new log
I20260808 02:22:27.433584 21107 ts_tablet_manager.cc:1397] T 29e46c96eaf143549b618ead37b157b0 P 99538ddf05824967b355a8b16c832793: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260808 02:22:27.433581 21109 tablet_bootstrap.cc:492] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5: No bootstrap required, opened a new log
I20260808 02:22:27.433806 21109 ts_tablet_manager.cc:1397] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 02:22:27.434190 21110 log.cc:824] T 29e46c96eaf143549b618ead37b157b0 P 17e00872e0e549a68917d157bad38c21: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:27.437652 21110 tablet_bootstrap.cc:492] T 29e46c96eaf143549b618ead37b157b0 P 17e00872e0e549a68917d157bad38c21: No bootstrap required, opened a new log
I20260808 02:22:27.437978 21110 ts_tablet_manager.cc:1397] T 29e46c96eaf143549b618ead37b157b0 P 17e00872e0e549a68917d157bad38c21: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260808 02:22:27.438045 21107 raft_consensus.cc:348] T 29e46c96eaf143549b618ead37b157b0 P 99538ddf05824967b355a8b16c832793 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99538ddf05824967b355a8b16c832793" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 36041 } } peers { permanent_uuid: "85207e75b51e46cfaf374c194f3ad1b5" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 43579 } } peers { permanent_uuid: "17e00872e0e549a68917d157bad38c21" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 45841 } }
I20260808 02:22:27.438043 21109 raft_consensus.cc:348] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99538ddf05824967b355a8b16c832793" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 36041 } } peers { permanent_uuid: "85207e75b51e46cfaf374c194f3ad1b5" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 43579 } } peers { permanent_uuid: "17e00872e0e549a68917d157bad38c21" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 45841 } }
I20260808 02:22:27.438370 21107 raft_consensus.cc:374] T 29e46c96eaf143549b618ead37b157b0 P 99538ddf05824967b355a8b16c832793 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:27.438370 21109 raft_consensus.cc:374] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:27.438434 21107 raft_consensus.cc:729] T 29e46c96eaf143549b618ead37b157b0 P 99538ddf05824967b355a8b16c832793 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99538ddf05824967b355a8b16c832793, State: Initialized, Role: FOLLOWER
I20260808 02:22:27.438436 21109 raft_consensus.cc:729] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85207e75b51e46cfaf374c194f3ad1b5, State: Initialized, Role: FOLLOWER
I20260808 02:22:27.438998 21109 consensus_queue.cc:260] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99538ddf05824967b355a8b16c832793" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 36041 } } peers { permanent_uuid: "85207e75b51e46cfaf374c194f3ad1b5" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 43579 } } peers { permanent_uuid: "17e00872e0e549a68917d157bad38c21" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 45841 } }
I20260808 02:22:27.438997 21107 consensus_queue.cc:260] T 29e46c96eaf143549b618ead37b157b0 P 99538ddf05824967b355a8b16c832793 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99538ddf05824967b355a8b16c832793" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 36041 } } peers { permanent_uuid: "85207e75b51e46cfaf374c194f3ad1b5" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 43579 } } peers { permanent_uuid: "17e00872e0e549a68917d157bad38c21" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 45841 } }
I20260808 02:22:27.440137 21078 heartbeater.cc:503] Master 127.24.79.254:38879 was elected leader, sending a full tablet report...
I20260808 02:22:27.440294 21107 ts_tablet_manager.cc:1428] T 29e46c96eaf143549b618ead37b157b0 P 99538ddf05824967b355a8b16c832793: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260808 02:22:27.440300 21109 ts_tablet_manager.cc:1428] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 02:22:27.440919 20675 heartbeater.cc:503] Master 127.24.79.254:38879 was elected leader, sending a full tablet report...
I20260808 02:22:27.442466 21110 raft_consensus.cc:348] T 29e46c96eaf143549b618ead37b157b0 P 17e00872e0e549a68917d157bad38c21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99538ddf05824967b355a8b16c832793" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 36041 } } peers { permanent_uuid: "85207e75b51e46cfaf374c194f3ad1b5" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 43579 } } peers { permanent_uuid: "17e00872e0e549a68917d157bad38c21" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 45841 } }
I20260808 02:22:27.443153 21110 raft_consensus.cc:374] T 29e46c96eaf143549b618ead37b157b0 P 17e00872e0e549a68917d157bad38c21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:27.443248 21110 raft_consensus.cc:729] T 29e46c96eaf143549b618ead37b157b0 P 17e00872e0e549a68917d157bad38c21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e00872e0e549a68917d157bad38c21, State: Initialized, Role: FOLLOWER
I20260808 02:22:27.443775 21110 consensus_queue.cc:260] T 29e46c96eaf143549b618ead37b157b0 P 17e00872e0e549a68917d157bad38c21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99538ddf05824967b355a8b16c832793" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 36041 } } peers { permanent_uuid: "85207e75b51e46cfaf374c194f3ad1b5" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 43579 } } peers { permanent_uuid: "17e00872e0e549a68917d157bad38c21" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 45841 } }
I20260808 02:22:27.444623 20872 heartbeater.cc:503] Master 127.24.79.254:38879 was elected leader, sending a full tablet report...
I20260808 02:22:27.445068 21110 ts_tablet_manager.cc:1428] T 29e46c96eaf143549b618ead37b157b0 P 17e00872e0e549a68917d157bad38c21: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20260808 02:22:27.451495 21115 raft_consensus.cc:482] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:22:27.451860 21115 raft_consensus.cc:504] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99538ddf05824967b355a8b16c832793" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 36041 } } peers { permanent_uuid: "85207e75b51e46cfaf374c194f3ad1b5" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 43579 } } peers { permanent_uuid: "17e00872e0e549a68917d157bad38c21" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 45841 } }
I20260808 02:22:27.453356 21115 leader_election.cc:288] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99538ddf05824967b355a8b16c832793 (127.24.79.193:36041), 17e00872e0e549a68917d157bad38c21 (127.24.79.194:45841)
I20260808 02:22:27.457180 21080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:27.453652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:36041 (local address 127.24.79.195:43993)
0808 02:22:27.453785 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:27.453798 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:27.453910 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:27.454168 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:27.454172 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:27.454198 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:27.454338 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:27.454346 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.455198 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.455202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:27.456026 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:27.456036 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.456735 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.456739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:27.456853 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:27.456876 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:22:27.456945 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 02:22:27.457029 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 02:22:27.457180 21100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:27.453767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:43993 (local address 127.24.79.193:36041)
0808 02:22:27.453911 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:27.453916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:27.453931 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:22:27.453992 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:27.453995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:27.454047 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:27.454123 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:27.454475 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.455023 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.456173 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.456614 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.456734 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:27.456797 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:27.456979 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 02:22:27.457073 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 02:22:27.457911 21119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:27.453962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:45841 (local address 127.24.79.195:50673)
0808 02:22:27.454434 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:27.454445 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:27.454549 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:27.454771 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:27.454774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:27.454792 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:27.454951 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:27.454958 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.455680 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.455683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:27.456451 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:27.456460 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.457246 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.457250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:27.457603 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:27.457624 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 02:22:27.457695 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 02:22:27.457766 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":183,"threads_started":1}
I20260808 02:22:27.457911 21104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:27.454126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:50673 (local address 127.24.79.194:45841)
0808 02:22:27.454261 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:27.454266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:27.454535 (+ 269us) server_negotiation.cc:407] Connection header received
0808 02:22:27.454641 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:27.454643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:27.454687 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:27.454751 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:27.455075 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.455570 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.456605 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.457125 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.457241 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:27.457294 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:27.457727 (+ 433us) server_negotiation.cc:299] Negotiation successful
0808 02:22:27.457806 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 02:22:27.457998 20610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29e46c96eaf143549b618ead37b157b0" candidate_uuid: "85207e75b51e46cfaf374c194f3ad1b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99538ddf05824967b355a8b16c832793" is_pre_election: true
I20260808 02:22:27.458395 20610 raft_consensus.cc:2454] T 29e46c96eaf143549b618ead37b157b0 P 99538ddf05824967b355a8b16c832793 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85207e75b51e46cfaf374c194f3ad1b5 in term 0.
I20260808 02:22:27.459012 20904 leader_election.cc:302] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85207e75b51e46cfaf374c194f3ad1b5, 99538ddf05824967b355a8b16c832793; no voters:
I20260808 02:22:27.459357 21115 raft_consensus.cc:2781] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:22:27.459383 20803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29e46c96eaf143549b618ead37b157b0" candidate_uuid: "85207e75b51e46cfaf374c194f3ad1b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e00872e0e549a68917d157bad38c21" is_pre_election: true
I20260808 02:22:27.459481 21115 raft_consensus.cc:482] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:22:27.459542 21115 raft_consensus.cc:3037] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:27.459719 20803 raft_consensus.cc:2454] T 29e46c96eaf143549b618ead37b157b0 P 17e00872e0e549a68917d157bad38c21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85207e75b51e46cfaf374c194f3ad1b5 in term 0.
I20260808 02:22:27.460814 21115 raft_consensus.cc:504] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99538ddf05824967b355a8b16c832793" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 36041 } } peers { permanent_uuid: "85207e75b51e46cfaf374c194f3ad1b5" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 43579 } } peers { permanent_uuid: "17e00872e0e549a68917d157bad38c21" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 45841 } }
I20260808 02:22:27.461158 21115 leader_election.cc:288] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5 [CANDIDATE]: Term 1 election: Requested vote from peers 99538ddf05824967b355a8b16c832793 (127.24.79.193:36041), 17e00872e0e549a68917d157bad38c21 (127.24.79.194:45841)
I20260808 02:22:27.461635 20803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29e46c96eaf143549b618ead37b157b0" candidate_uuid: "85207e75b51e46cfaf374c194f3ad1b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e00872e0e549a68917d157bad38c21"
I20260808 02:22:27.461650 20610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29e46c96eaf143549b618ead37b157b0" candidate_uuid: "85207e75b51e46cfaf374c194f3ad1b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99538ddf05824967b355a8b16c832793"
I20260808 02:22:27.461784 20803 raft_consensus.cc:3037] T 29e46c96eaf143549b618ead37b157b0 P 17e00872e0e549a68917d157bad38c21 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:27.461839 20610 raft_consensus.cc:3037] T 29e46c96eaf143549b618ead37b157b0 P 99538ddf05824967b355a8b16c832793 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:27.463017 20803 raft_consensus.cc:2454] T 29e46c96eaf143549b618ead37b157b0 P 17e00872e0e549a68917d157bad38c21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85207e75b51e46cfaf374c194f3ad1b5 in term 1.
I20260808 02:22:27.463361 20610 raft_consensus.cc:2454] T 29e46c96eaf143549b618ead37b157b0 P 99538ddf05824967b355a8b16c832793 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85207e75b51e46cfaf374c194f3ad1b5 in term 1.
I20260808 02:22:27.463728 20904 leader_election.cc:302] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85207e75b51e46cfaf374c194f3ad1b5, 99538ddf05824967b355a8b16c832793; no voters:
I20260808 02:22:27.463935 21115 raft_consensus.cc:2781] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:27.464196 21115 raft_consensus.cc:686] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5 [term 1 LEADER]: Becoming Leader. State: Replica: 85207e75b51e46cfaf374c194f3ad1b5, State: Running, Role: LEADER
I20260808 02:22:27.464612 21115 consensus_queue.cc:237] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99538ddf05824967b355a8b16c832793" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 36041 } } peers { permanent_uuid: "85207e75b51e46cfaf374c194f3ad1b5" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 43579 } } peers { permanent_uuid: "17e00872e0e549a68917d157bad38c21" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 45841 } }
I20260808 02:22:27.468041 20392 catalog_manager.cc:5495] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 85207e75b51e46cfaf374c194f3ad1b5 (127.24.79.195). New cstate: current_term: 1 leader_uuid: "85207e75b51e46cfaf374c194f3ad1b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99538ddf05824967b355a8b16c832793" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 36041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85207e75b51e46cfaf374c194f3ad1b5" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 43579 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17e00872e0e549a68917d157bad38c21" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 45841 } health_report { overall_health: UNKNOWN } } }
I20260808 02:22:27.487141 21102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:27.483024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37314 (local address 127.24.79.195:43579)
0808 02:22:27.483206 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:27.483210 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:27.483224 (+ 14us) server_negotiation.cc:407] Connection header received
0808 02:22:27.483325 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:27.483328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:27.483387 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:27.483481 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:22:27.483851 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.484399 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.485354 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.485591 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.485699 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:27.486082 (+ 383us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:22:27.486604 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:22:27.486749 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:27.486827 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 02:22:27.486906 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 02:22:27.495425 20610 raft_consensus.cc:1261] T 29e46c96eaf143549b618ead37b157b0 P 99538ddf05824967b355a8b16c832793 [term 1 FOLLOWER]: Refusing update from remote peer 85207e75b51e46cfaf374c194f3ad1b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:22:27.495450 20803 raft_consensus.cc:1261] T 29e46c96eaf143549b618ead37b157b0 P 17e00872e0e549a68917d157bad38c21 [term 1 FOLLOWER]: Refusing update from remote peer 85207e75b51e46cfaf374c194f3ad1b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:22:27.496335 21115 consensus_queue.cc:1035] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99538ddf05824967b355a8b16c832793" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 36041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:22:27.496634 21120 consensus_queue.cc:1035] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e00872e0e549a68917d157bad38c21" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:22:27.508203 21125 mvcc.cc:205] Tried to move back new op lower bound from 7316093941729513472 to 7316093941618135040. Current Snapshot: MvccSnapshot[applied={T|T < 7316093941729513472}]
I20260808 02:22:27.508699 21126 mvcc.cc:205] Tried to move back new op lower bound from 7316093941729513472 to 7316093941618135040. Current Snapshot: MvccSnapshot[applied={T|T < 7316093941729513472}]
I20260808 02:22:27.517789 20392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:22:27.518124 20392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:22:27.521780 20392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6977df97d93b4cf9a812129dda2072c5: Sending DeleteTablet for 3 replicas of tablet 29e46c96eaf143549b618ead37b157b0
I20260808 02:22:27.522944 20581 tablet_service.cc:1514] Processing DeleteTablet for tablet 29e46c96eaf143549b618ead37b157b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:22:27 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:37566
I20260808 02:22:27.522944 20769 tablet_service.cc:1514] Processing DeleteTablet for tablet 29e46c96eaf143549b618ead37b157b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:22:27 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:35930
I20260808 02:22:27.522946 20971 tablet_service.cc:1514] Processing DeleteTablet for tablet 29e46c96eaf143549b618ead37b157b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:22:27 UTC) from {username='kudu', principal='kudu/127.24.79.254@KRBTEST.COM'} at 127.0.0.1:37310
I20260808 02:22:27.523634 21138 tablet_replica.cc:330] T 29e46c96eaf143549b618ead37b157b0 P 17e00872e0e549a68917d157bad38c21: stopping tablet replica
I20260808 02:22:27.523833 21141 tablet_replica.cc:330] T 29e46c96eaf143549b618ead37b157b0 P 99538ddf05824967b355a8b16c832793: stopping tablet replica
I20260808 02:22:27.524091 21138 raft_consensus.cc:2229] T 29e46c96eaf143549b618ead37b157b0 P 17e00872e0e549a68917d157bad38c21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:22:27.524179 21141 raft_consensus.cc:2229] T 29e46c96eaf143549b618ead37b157b0 P 99538ddf05824967b355a8b16c832793 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:22:27.524525 21138 raft_consensus.cc:2258] T 29e46c96eaf143549b618ead37b157b0 P 17e00872e0e549a68917d157bad38c21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:22:27.524559 21141 raft_consensus.cc:2258] T 29e46c96eaf143549b618ead37b157b0 P 99538ddf05824967b355a8b16c832793 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:22:27.524675 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 20485
I20260808 02:22:27.525003 21139 tablet_replica.cc:330] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5: stopping tablet replica
I20260808 02:22:27.525353 21139 raft_consensus.cc:2229] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:22:27.525955 21139 raft_consensus.cc:2258] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:22:27.533279 21138 ts_tablet_manager.cc:1905] T 29e46c96eaf143549b618ead37b157b0 P 17e00872e0e549a68917d157bad38c21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:22:27.536595 21139 ts_tablet_manager.cc:1905] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:22:27.537307 21138 ts_tablet_manager.cc:1918] T 29e46c96eaf143549b618ead37b157b0 P 17e00872e0e549a68917d157bad38c21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:22:27.537449 21138 log.cc:1196] T 29e46c96eaf143549b618ead37b157b0 P 17e00872e0e549a68917d157bad38c21: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/29e46c96eaf143549b618ead37b157b0
I20260808 02:22:27.537811 21138 ts_tablet_manager.cc:1939] T 29e46c96eaf143549b618ead37b157b0 P 17e00872e0e549a68917d157bad38c21: Deleting consensus metadata
I20260808 02:22:27.539495 20374 catalog_manager.cc:4841] TS 17e00872e0e549a68917d157bad38c21 (127.24.79.194:45841): tablet 29e46c96eaf143549b618ead37b157b0 (table test-table [id=d34383ff1a174a1eb1713e91b0b9cc14]) successfully deleted
I20260808 02:22:27.540155 21139 ts_tablet_manager.cc:1918] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:22:27.540282 21139 log.cc:1196] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/29e46c96eaf143549b618ead37b157b0
I20260808 02:22:27.540630 21139 ts_tablet_manager.cc:1939] T 29e46c96eaf143549b618ead37b157b0 P 85207e75b51e46cfaf374c194f3ad1b5: Deleting consensus metadata
I20260808 02:22:27.542321 20372 catalog_manager.cc:4841] TS 85207e75b51e46cfaf374c194f3ad1b5 (127.24.79.195:43579): tablet 29e46c96eaf143549b618ead37b157b0 (table test-table [id=d34383ff1a174a1eb1713e91b0b9cc14]) successfully deleted
W20260808 02:22:27.544333 20374 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.79.193:36041 (error 108)
I20260808 02:22:27.545327 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 20680
I20260808 02:22:27.545652 21103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:27.545287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:36041 (local address 127.0.0.1:37582)
0808 02:22:27.545457 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:27.545551 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.193:36041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260808 02:22:27.562354 20374 catalog_manager.cc:4584] TS 99538ddf05824967b355a8b16c832793 (127.24.79.193:36041): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 29e46c96eaf143549b618ead37b157b0: Network error: Client connection negotiation failed: client connection to 127.24.79.193:36041: connect: Connection refused (error 111)
I20260808 02:22:27.572850 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 20880
I20260808 02:22:27.594249 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 20352
2026-08-08T02:22:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2686 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T02:22:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:27Z Disabled control of system clock
I20260808 02:22:27.636838 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:38025
--webserver_interface=127.24.79.254
--webserver_port=0
--builtin_ntp_servers=127.24.79.212:38105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:38025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:22:27.767077 21146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:27.767442 21146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:27.767485 21146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:27.771986 21146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:27.772104 21146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:27.772130 21146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:27.772149 21146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:27.772167 21146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:27.777980 21146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:38105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:38025
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:38025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:27.779373 21146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:27.780980 21146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:27.790714 21156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:27.790724 21153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:27.790710 21154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:27.791229 21146 server_base.cc:1034] running on GCE node
I20260808 02:22:27.791893 21146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:27.793241 21146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:27.794516 21146 hybrid_clock.cc:648] HybridClock initialized: now 1786155747794479 us; error 54 us; skew 500 ppm
I20260808 02:22:27.797005 21146 webserver.cc:460] Webserver started at http://127.24.79.254:39791/ using document root <none> and password file <none>
I20260808 02:22:27.797717 21146 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:27.797802 21146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:27.798005 21146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:27.800539 21146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "9a209582d179489a816996c393c237b4"
format_stamp: "Formatted at 2026-08-08 02:22:27 on dist-test-slave-1vmg"
server_key: "6741516a4a00b305f93bad16d63b8612"
server_key_iv: "1af974258bacc6fe9aaed95450143dbc"
server_key_version: "encryptionkey@0"
I20260808 02:22:27.801208 21146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "9a209582d179489a816996c393c237b4"
format_stamp: "Formatted at 2026-08-08 02:22:27 on dist-test-slave-1vmg"
server_key: "6741516a4a00b305f93bad16d63b8612"
server_key_iv: "1af974258bacc6fe9aaed95450143dbc"
server_key_version: "encryptionkey@0"
I20260808 02:22:27.805836 21146 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:22:27.808966 21166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:27.810585 21146 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:22:27.810777 21146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "9a209582d179489a816996c393c237b4"
format_stamp: "Formatted at 2026-08-08 02:22:27 on dist-test-slave-1vmg"
server_key: "6741516a4a00b305f93bad16d63b8612"
server_key_iv: "1af974258bacc6fe9aaed95450143dbc"
server_key_version: "encryptionkey@0"
I20260808 02:22:27.810961 21146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:27.830699 21146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:27.831643 21146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:27.831872 21146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:27.839844 21241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:38025 every 8 connection(s)
I20260808 02:22:27.839844 21146 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:38025
I20260808 02:22:27.841192 21146 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:27.845002 21242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:27.846671 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 21146
I20260808 02:22:27.846843 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:27.847252 24895 external_mini_cluster.cc:1428] Setting key 4d6b7b40602a992fd311873cfc11ac38
I20260808 02:22:27.852980 21242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a209582d179489a816996c393c237b4: Bootstrap starting.
I20260808 02:22:27.855798 21242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a209582d179489a816996c393c237b4: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:27.856721 21242 log.cc:824] T 00000000000000000000000000000000 P 9a209582d179489a816996c393c237b4: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:27.858484 21246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:27.849067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37760 (local address 127.24.79.254:38025)
0808 02:22:27.849917 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:27.849937 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:27.849979 (+ 42us) server_negotiation.cc:407] Connection header received
0808 02:22:27.850762 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:27.850786 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:27.851342 (+ 556us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:27.851725 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:27.852229 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.853450 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.854395 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:27.854957 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:27.855146 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:27.856214 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:27.856220 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:27.856239 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:27.856698 (+ 459us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:27.856769 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:27.856792 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:27.856904 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:27.857085 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 02:22:27.857628 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":170,"threads_started":1}
I20260808 02:22:27.859313 21242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a209582d179489a816996c393c237b4: No bootstrap required, opened a new log
I20260808 02:22:27.863253 21242 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a209582d179489a816996c393c237b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a209582d179489a816996c393c237b4" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38025 } }
I20260808 02:22:27.863564 21242 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a209582d179489a816996c393c237b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:27.863631 21242 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a209582d179489a816996c393c237b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a209582d179489a816996c393c237b4, State: Initialized, Role: FOLLOWER
I20260808 02:22:27.864107 21242 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a209582d179489a816996c393c237b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a209582d179489a816996c393c237b4" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38025 } }
I20260808 02:22:27.864261 21242 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a209582d179489a816996c393c237b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:27.864336 21242 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a209582d179489a816996c393c237b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:27.864447 21242 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a209582d179489a816996c393c237b4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:27.865739 21242 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a209582d179489a816996c393c237b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a209582d179489a816996c393c237b4" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38025 } }
I20260808 02:22:27.866147 21242 leader_election.cc:302] T 00000000000000000000000000000000 P 9a209582d179489a816996c393c237b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a209582d179489a816996c393c237b4; no voters:
I20260808 02:22:27.866514 21242 leader_election.cc:288] T 00000000000000000000000000000000 P 9a209582d179489a816996c393c237b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:22:27.866685 21248 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a209582d179489a816996c393c237b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:27.866962 21248 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a209582d179489a816996c393c237b4 [term 1 LEADER]: Becoming Leader. State: Replica: 9a209582d179489a816996c393c237b4, State: Running, Role: LEADER
I20260808 02:22:27.867403 21248 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a209582d179489a816996c393c237b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a209582d179489a816996c393c237b4" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38025 } }
I20260808 02:22:27.867990 21242 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a209582d179489a816996c393c237b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:27.869383 21249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a209582d179489a816996c393c237b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a209582d179489a816996c393c237b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a209582d179489a816996c393c237b4" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38025 } } }
I20260808 02:22:27.869400 21251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a209582d179489a816996c393c237b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a209582d179489a816996c393c237b4. Latest consensus state: current_term: 1 leader_uuid: "9a209582d179489a816996c393c237b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a209582d179489a816996c393c237b4" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38025 } } }
I20260808 02:22:27.869603 21251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a209582d179489a816996c393c237b4 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:27.869880 21249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a209582d179489a816996c393c237b4 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:27.870316 21264 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:22:27.874125 21264 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:22:27.881201 21264 catalog_manager.cc:1347] Generated new cluster ID: c525858007ae4aaa96386a6d85c6b646
I20260808 02:22:27.881300 21264 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:22:27.901636 21264 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:22:27.903195 21264 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:22:27.909907 21264 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a209582d179489a816996c393c237b4: Generated new TSK 0
I20260808 02:22:27.911038 21264 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:22:27.923530 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.193:0
--local_ip_for_outbound_sockets=127.24.79.193
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:38025
--builtin_ntp_servers=127.24.79.212:38105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:22:28.059815 21278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:28.060184 21278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:28.060243 21278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:28.064949 21278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:28.065053 21278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:28.065140 21278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.193
I20260808 02:22:28.070863 21278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:38105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.79.193
--webserver_port=0
--tserver_master_addrs=127.24.79.254:38025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.193
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:28.072341 21278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:28.073977 21278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:28.085579 21288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:28.085598 21291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:28.085582 21287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:28.085889 21278 server_base.cc:1034] running on GCE node
I20260808 02:22:28.086540 21278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:28.087386 21278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:28.088624 21278 hybrid_clock.cc:648] HybridClock initialized: now 1786155748088577 us; error 62 us; skew 500 ppm
I20260808 02:22:28.091246 21278 webserver.cc:460] Webserver started at http://127.24.79.193:36175/ using document root <none> and password file <none>
I20260808 02:22:28.091929 21278 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:28.092008 21278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:28.092231 21278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:28.094635 21278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-0/data/instance:
uuid: "0205d6e78d5d4a2194e1b8ce27560aa9"
format_stamp: "Formatted at 2026-08-08 02:22:28 on dist-test-slave-1vmg"
server_key: "7c10126012984bac3d46129859ba9ecd"
server_key_iv: "ed90a714f1c00e5bb7f9131bd8f3154f"
server_key_version: "encryptionkey@0"
I20260808 02:22:28.095304 21278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance:
uuid: "0205d6e78d5d4a2194e1b8ce27560aa9"
format_stamp: "Formatted at 2026-08-08 02:22:28 on dist-test-slave-1vmg"
server_key: "7c10126012984bac3d46129859ba9ecd"
server_key_iv: "ed90a714f1c00e5bb7f9131bd8f3154f"
server_key_version: "encryptionkey@0"
I20260808 02:22:28.099830 21278 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:22:28.103044 21301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:28.104672 21278 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:22:28.104869 21278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-0/wal
uuid: "0205d6e78d5d4a2194e1b8ce27560aa9"
format_stamp: "Formatted at 2026-08-08 02:22:28 on dist-test-slave-1vmg"
server_key: "7c10126012984bac3d46129859ba9ecd"
server_key_iv: "ed90a714f1c00e5bb7f9131bd8f3154f"
server_key_version: "encryptionkey@0"
I20260808 02:22:28.105006 21278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:28.118988 21278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:28.119858 21278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:28.120064 21278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:28.120800 21278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:22:28.121938 21278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:22:28.121994 21278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:28.122057 21278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:22:28.122085 21278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:28.132005 21278 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.193:39341
I20260808 02:22:28.132030 21471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.193:39341 every 8 connection(s)
I20260808 02:22:28.133090 21278 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-0/data/info.pb
I20260808 02:22:28.133553 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 21278
I20260808 02:22:28.133746 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-0/wal/instance
I20260808 02:22:28.134104 24895 external_mini_cluster.cc:1428] Setting key 563a384a38b26186176c38b27390b4e7
I20260808 02:22:28.136389 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.194:0
--local_ip_for_outbound_sockets=127.24.79.194
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:38025
--builtin_ntp_servers=127.24.79.212:38105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 02:22:28.145021 21246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:28.135453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.193:38231 (local address 127.24.79.254:38025)
0808 02:22:28.135602 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:28.135606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:28.137050 (+ 1444us) server_negotiation.cc:407] Connection header received
0808 02:22:28.138560 (+ 1510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:28.138564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:28.138638 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:28.138725 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:28.139973 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.140737 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.142220 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.142588 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.142737 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:28.144300 (+ 1563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:28.144305 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:28.144308 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:28.144329 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:28.144362 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:28.144375 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:28.144480 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:28.144724 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 02:22:28.144830 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:22:28.146102 21477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:28.136035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38025 (local address 127.24.79.193:38231)
0808 02:22:28.136861 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:28.136924 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:28.138293 (+ 1369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:28.139004 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:28.139014 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:28.139121 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:28.139782 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:28.139799 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.140947 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.140951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:28.142013 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:28.142032 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.142857 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.142861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:28.143141 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:28.143882 (+ 741us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:22:28.143884 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:28.143943 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:22:28.144052 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:28.144514 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:28.144528 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:28.144536 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 02:22:28.144697 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 02:22:28.145028 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":131,"threads_started":1}
I20260808 02:22:28.148142 21474 heartbeater.cc:347] Connected to a master server at 127.24.79.254:38025
I20260808 02:22:28.148528 21474 heartbeater.cc:464] Registering TS with master...
I20260808 02:22:28.149459 21474 heartbeater.cc:511] Master 127.24.79.254:38025 requested a full tablet report, sending...
I20260808 02:22:28.151767 21185 ts_manager.cc:194] Registered new tserver with Master: 0205d6e78d5d4a2194e1b8ce27560aa9 (127.24.79.193:39341)
I20260808 02:22:28.153165 21185 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.79.193:38231
W20260808 02:22:28.267799 21478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:28.268136 21478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:28.268178 21478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:28.273023 21478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:28.273128 21478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:28.273190 21478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.194
I20260808 02:22:28.278690 21478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:38105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.79.194
--webserver_port=0
--tserver_master_addrs=127.24.79.254:38025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.194
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:28.280103 21478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:28.281695 21478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:28.293282 21490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:28.293278 21493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:28.293269 21487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:28.293777 21478 server_base.cc:1034] running on GCE node
I20260808 02:22:28.294380 21478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:28.295316 21478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:28.296566 21478 hybrid_clock.cc:648] HybridClock initialized: now 1786155748296540 us; error 34 us; skew 500 ppm
I20260808 02:22:28.299059 21478 webserver.cc:460] Webserver started at http://127.24.79.194:45439/ using document root <none> and password file <none>
I20260808 02:22:28.299786 21478 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:28.299872 21478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:28.300076 21478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:28.302357 21478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-1/data/instance:
uuid: "a8202c736da24a6eb4b0bd6c682b3557"
format_stamp: "Formatted at 2026-08-08 02:22:28 on dist-test-slave-1vmg"
server_key: "b3136f45c27fe264291fe044b7b9058f"
server_key_iv: "550dc4b676368fd68373a63d87005649"
server_key_version: "encryptionkey@0"
I20260808 02:22:28.303016 21478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance:
uuid: "a8202c736da24a6eb4b0bd6c682b3557"
format_stamp: "Formatted at 2026-08-08 02:22:28 on dist-test-slave-1vmg"
server_key: "b3136f45c27fe264291fe044b7b9058f"
server_key_iv: "550dc4b676368fd68373a63d87005649"
server_key_version: "encryptionkey@0"
I20260808 02:22:28.307369 21478 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:22:28.310428 21503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:28.312232 21478 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:22:28.312402 21478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-1/wal
uuid: "a8202c736da24a6eb4b0bd6c682b3557"
format_stamp: "Formatted at 2026-08-08 02:22:28 on dist-test-slave-1vmg"
server_key: "b3136f45c27fe264291fe044b7b9058f"
server_key_iv: "550dc4b676368fd68373a63d87005649"
server_key_version: "encryptionkey@0"
I20260808 02:22:28.312549 21478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:28.348521 21478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:28.349385 21478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:28.349587 21478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:28.350288 21478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:22:28.351508 21478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:22:28.351565 21478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:28.351629 21478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:22:28.351656 21478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:28.361995 21478 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.194:33433
I20260808 02:22:28.362064 21680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.194:33433 every 8 connection(s)
I20260808 02:22:28.363134 21478 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-1/data/info.pb
I20260808 02:22:28.366992 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 21478
I20260808 02:22:28.367152 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-1/wal/instance
I20260808 02:22:28.367461 24895 external_mini_cluster.cc:1428] Setting key 9939456fe855c84e0335ca6e9d932fa5
I20260808 02:22:28.370159 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.79.195:0
--local_ip_for_outbound_sockets=127.24.79.195
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:38025
--builtin_ntp_servers=127.24.79.212:38105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 02:22:28.374804 21246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:28.365175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.194:58313 (local address 127.24.79.254:38025)
0808 02:22:28.365367 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:28.365372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:28.366635 (+ 1263us) server_negotiation.cc:407] Connection header received
0808 02:22:28.368636 (+ 2001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:28.368640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:28.368762 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:28.368848 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:28.370335 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.370959 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.372587 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.372924 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.373049 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:28.374280 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:28.374284 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:28.374287 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:28.374306 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:28.374332 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:28.374342 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:28.374419 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:28.374551 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 02:22:28.374640 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 02:22:28.375895 21684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:28.365679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38025 (local address 127.24.79.194:58313)
0808 02:22:28.366466 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:28.366514 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:28.368288 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:28.369121 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:28.369132 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:28.369267 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:28.370118 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:28.370147 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.371100 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.371104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:28.372418 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:28.372431 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.373093 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.373096 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:28.373278 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:28.373952 (+ 674us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:22:28.373953 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:28.374005 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:22:28.374090 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:28.374422 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:28.374428 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:28.374433 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 02:22:28.374534 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 02:22:28.374861 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":135,"threads_started":1}
I20260808 02:22:28.377414 21681 heartbeater.cc:347] Connected to a master server at 127.24.79.254:38025
I20260808 02:22:28.377710 21681 heartbeater.cc:464] Registering TS with master...
I20260808 02:22:28.378338 21681 heartbeater.cc:511] Master 127.24.79.254:38025 requested a full tablet report, sending...
I20260808 02:22:28.379712 21185 ts_manager.cc:194] Registered new tserver with Master: a8202c736da24a6eb4b0bd6c682b3557 (127.24.79.194:33433)
I20260808 02:22:28.380395 21185 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.79.194:58313
W20260808 02:22:28.498049 21685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:28.498361 21685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:28.498404 21685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:28.502615 21685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:28.502722 21685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:28.502781 21685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.79.195
I20260808 02:22:28.507951 21685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:38105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.79.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.79.195
--webserver_port=0
--tserver_master_addrs=127.24.79.254:38025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.79.195
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:28.509255 21685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:28.510807 21685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:28.524326 21695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:28.524343 21698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:28.524430 21685 server_base.cc:1034] running on GCE node
W20260808 02:22:28.524346 21696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:28.525195 21685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:28.526044 21685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:28.527298 21685 hybrid_clock.cc:648] HybridClock initialized: now 1786155748527280 us; error 36 us; skew 500 ppm
I20260808 02:22:28.529670 21685 webserver.cc:460] Webserver started at http://127.24.79.195:42341/ using document root <none> and password file <none>
I20260808 02:22:28.530328 21685 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:28.530407 21685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:28.530627 21685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:28.533519 21685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-2/data/instance:
uuid: "979b94eebb6a4b82b378cae65130da43"
format_stamp: "Formatted at 2026-08-08 02:22:28 on dist-test-slave-1vmg"
server_key: "41c387584bac693161555e2df3090376"
server_key_iv: "278f436e4517edbd77e61062cfff2430"
server_key_version: "encryptionkey@0"
I20260808 02:22:28.534340 21685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance:
uuid: "979b94eebb6a4b82b378cae65130da43"
format_stamp: "Formatted at 2026-08-08 02:22:28 on dist-test-slave-1vmg"
server_key: "41c387584bac693161555e2df3090376"
server_key_iv: "278f436e4517edbd77e61062cfff2430"
server_key_version: "encryptionkey@0"
I20260808 02:22:28.539302 21685 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 02:22:28.542704 21708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:28.544486 21685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:22:28.544649 21685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-2/wal
uuid: "979b94eebb6a4b82b378cae65130da43"
format_stamp: "Formatted at 2026-08-08 02:22:28 on dist-test-slave-1vmg"
server_key: "41c387584bac693161555e2df3090376"
server_key_iv: "278f436e4517edbd77e61062cfff2430"
server_key_version: "encryptionkey@0"
I20260808 02:22:28.544814 21685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:28.561105 21685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:28.561998 21685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:28.562216 21685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:28.563041 21685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:22:28.564236 21685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:22:28.564306 21685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:28.564354 21685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:22:28.564368 21685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:28.575511 21685 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.195:39413
I20260808 02:22:28.575538 21882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.195:39413 every 8 connection(s)
I20260808 02:22:28.576642 21685 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-2/data/info.pb
I20260808 02:22:28.580107 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 21685
I20260808 02:22:28.580346 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-2/wal/instance
I20260808 02:22:28.580751 24895 external_mini_cluster.cc:1428] Setting key 6be9ad726186431b4b7f7407d923295c
I20260808 02:22:28.589485 21246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:28.579053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:39627 (local address 127.24.79.254:38025)
0808 02:22:28.579239 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:28.579243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:28.580665 (+ 1422us) server_negotiation.cc:407] Connection header received
0808 02:22:28.582455 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:28.582459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:28.582525 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:28.582603 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:28.583869 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.584744 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.586478 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.586745 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.586851 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:28.588663 (+ 1812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:28.588667 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:28.588670 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:28.588687 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:28.588712 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:28.588722 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:28.588800 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:28.589260 (+ 460us) server_negotiation.cc:299] Negotiation successful
0808 02:22:28.589345 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:22:28.590555 21886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:28.579461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:38025 (local address 127.24.79.195:39627)
0808 02:22:28.580464 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:28.580534 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:28.582022 (+ 1488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:28.582855 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:28.582865 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:28.583029 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:28.583686 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:28.583703 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.585163 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.585168 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:28.586289 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:28.586304 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.586989 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.586995 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:28.587210 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:28.588107 (+ 897us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:22:28.588109 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:28.588168 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:22:28.588434 (+ 266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:28.588872 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:28.588886 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:28.588893 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 02:22:28.589166 (+ 273us) client_negotiation.cc:240] Negotiation successful
0808 02:22:28.589514 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":196,"threads_started":1}
I20260808 02:22:28.592393 21884 heartbeater.cc:347] Connected to a master server at 127.24.79.254:38025
I20260808 02:22:28.592743 21884 heartbeater.cc:464] Registering TS with master...
I20260808 02:22:28.593628 21884 heartbeater.cc:511] Master 127.24.79.254:38025 requested a full tablet report, sending...
I20260808 02:22:28.595007 21185 ts_manager.cc:194] Registered new tserver with Master: 979b94eebb6a4b82b378cae65130da43 (127.24.79.195:39413)
I20260808 02:22:28.595647 21185 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.79.195:39627
I20260808 02:22:28.608068 24895 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:22:28.615401 21246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:28.610505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37768 (local address 127.24.79.254:38025)
0808 02:22:28.610698 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:28.610702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:28.610977 (+ 275us) server_negotiation.cc:407] Connection header received
0808 02:22:28.611162 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:28.611165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:28.611263 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:28.611349 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:28.611826 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.612410 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.613374 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.613616 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.613709 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:28.614822 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:28.614825 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:28.614827 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:28.614844 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:28.614869 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:28.614877 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:28.615035 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:28.615156 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 02:22:28.615259 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 02:22:28.619784 21185 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:22:28.623183 21185 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:22:28.640580 21899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:28.634040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:33433 (local address 127.0.0.1:37110)
0808 02:22:28.635083 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:28.635095 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:28.635220 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:28.636237 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:28.636241 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:28.636260 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:28.636364 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:28.636371 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.637402 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.637406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:28.638766 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:28.638779 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.639916 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.639920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:28.640084 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:28.640143 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 02:22:28.640252 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 02:22:28.640401 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":981,"mutex_wait_us":17,"thread_start_us":136,"threads_started":1}
I20260808 02:22:28.640645 21902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:28.634651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37110 (local address 127.24.79.194:33433)
0808 02:22:28.635796 (+ 1145us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:28.635808 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:28.635832 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:22:28.635911 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:28.635915 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:28.636083 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:28.636238 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:28.636483 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.637282 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.638943 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.639742 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.640066 (+ 324us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:28.640296 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:28.640357 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 02:22:28.640469 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1020,"thread_start_us":136,"threads_started":1}
I20260808 02:22:28.640924 21903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:28.634724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.195:39413 (local address 127.0.0.1:53602)
0808 02:22:28.635448 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:28.635460 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:28.635534 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:28.636077 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:28.636079 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:28.636094 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:28.636182 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:28.636187 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.637752 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.637755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:28.639192 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:28.639203 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.640289 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.640293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:28.640375 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:28.640410 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 02:22:28.640468 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 02:22:28.640529 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"mutex_wait_us":207,"thread_start_us":120,"threads_started":1}
I20260808 02:22:28.640966 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:28.635160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53602 (local address 127.24.79.195:39413)
0808 02:22:28.635662 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:28.635674 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:28.635697 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:22:28.635766 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:28.635769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:28.635923 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:28.636335 (+ 412us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:28.636482 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.637607 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.639332 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.640159 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.640277 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:28.640444 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:28.640492 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 02:22:28.640579 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":171,"threads_started":1}
I20260808 02:22:28.641116 21900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:28.633857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:39341 (local address 127.0.0.1:41032)
0808 02:22:28.634985 (+ 1128us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:28.635048 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:28.635204 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:28.635965 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:28.635973 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:28.635999 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:28.636164 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:28.636172 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.637755 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.637758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:28.638947 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:28.638959 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.639895 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.639898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:28.640802 (+ 904us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:28.640827 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 02:22:28.640892 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 02:22:28.640965 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":887,"mutex_wait_us":157,"thread_start_us":119,"threads_started":1}
I20260808 02:22:28.641507 21901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:28.634040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41032 (local address 127.24.79.193:39341)
0808 02:22:28.634509 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:28.634517 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:28.635359 (+ 842us) server_negotiation.cc:407] Connection header received
0808 02:22:28.635452 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:28.635459 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:28.635775 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:28.635953 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:28.636311 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.637637 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.639083 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.639740 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.639867 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:28.640116 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:28.641279 (+ 1163us) server_negotiation.cc:299] Negotiation successful
0808 02:22:28.641381 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":143,"threads_started":1}
I20260808 02:22:28.644412 21363 tablet_service.cc:1467] Processing CreateTablet for tablet 86c3df6f408f4947a444e697c31736a8 (DEFAULT_TABLE table=test-table [id=a7379b2603844adc8ed4506b557f5a53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:22:28.644434 21789 tablet_service.cc:1467] Processing CreateTablet for tablet 86c3df6f408f4947a444e697c31736a8 (DEFAULT_TABLE table=test-table [id=a7379b2603844adc8ed4506b557f5a53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:22:28.644425 21574 tablet_service.cc:1467] Processing CreateTablet for tablet 86c3df6f408f4947a444e697c31736a8 (DEFAULT_TABLE table=test-table [id=a7379b2603844adc8ed4506b557f5a53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:22:28.645673 21363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86c3df6f408f4947a444e697c31736a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:28.645673 21574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86c3df6f408f4947a444e697c31736a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:28.645673 21789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86c3df6f408f4947a444e697c31736a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:28.653131 21905 tablet_bootstrap.cc:492] T 86c3df6f408f4947a444e697c31736a8 P 0205d6e78d5d4a2194e1b8ce27560aa9: Bootstrap starting.
I20260808 02:22:28.654954 21906 tablet_bootstrap.cc:492] T 86c3df6f408f4947a444e697c31736a8 P a8202c736da24a6eb4b0bd6c682b3557: Bootstrap starting.
I20260808 02:22:28.656519 21905 tablet_bootstrap.cc:654] T 86c3df6f408f4947a444e697c31736a8 P 0205d6e78d5d4a2194e1b8ce27560aa9: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:28.657374 21906 tablet_bootstrap.cc:654] T 86c3df6f408f4947a444e697c31736a8 P a8202c736da24a6eb4b0bd6c682b3557: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:28.657553 21905 log.cc:824] T 86c3df6f408f4947a444e697c31736a8 P 0205d6e78d5d4a2194e1b8ce27560aa9: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:28.658284 21906 log.cc:824] T 86c3df6f408f4947a444e697c31736a8 P a8202c736da24a6eb4b0bd6c682b3557: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:28.658582 21907 tablet_bootstrap.cc:492] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43: Bootstrap starting.
I20260808 02:22:28.660365 21905 tablet_bootstrap.cc:492] T 86c3df6f408f4947a444e697c31736a8 P 0205d6e78d5d4a2194e1b8ce27560aa9: No bootstrap required, opened a new log
I20260808 02:22:28.660631 21905 ts_tablet_manager.cc:1397] T 86c3df6f408f4947a444e697c31736a8 P 0205d6e78d5d4a2194e1b8ce27560aa9: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260808 02:22:28.661059 21907 tablet_bootstrap.cc:654] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:28.661259 21906 tablet_bootstrap.cc:492] T 86c3df6f408f4947a444e697c31736a8 P a8202c736da24a6eb4b0bd6c682b3557: No bootstrap required, opened a new log
I20260808 02:22:28.661484 21906 ts_tablet_manager.cc:1397] T 86c3df6f408f4947a444e697c31736a8 P a8202c736da24a6eb4b0bd6c682b3557: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 02:22:28.661929 21907 log.cc:824] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:28.664189 21907 tablet_bootstrap.cc:492] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43: No bootstrap required, opened a new log
I20260808 02:22:28.664448 21907 ts_tablet_manager.cc:1397] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 02:22:28.665556 21906 raft_consensus.cc:348] T 86c3df6f408f4947a444e697c31736a8 P a8202c736da24a6eb4b0bd6c682b3557 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0205d6e78d5d4a2194e1b8ce27560aa9" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 39341 } } peers { permanent_uuid: "a8202c736da24a6eb4b0bd6c682b3557" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 33433 } } peers { permanent_uuid: "979b94eebb6a4b82b378cae65130da43" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 39413 } }
I20260808 02:22:28.665556 21905 raft_consensus.cc:348] T 86c3df6f408f4947a444e697c31736a8 P 0205d6e78d5d4a2194e1b8ce27560aa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0205d6e78d5d4a2194e1b8ce27560aa9" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 39341 } } peers { permanent_uuid: "a8202c736da24a6eb4b0bd6c682b3557" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 33433 } } peers { permanent_uuid: "979b94eebb6a4b82b378cae65130da43" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 39413 } }
I20260808 02:22:28.665889 21906 raft_consensus.cc:374] T 86c3df6f408f4947a444e697c31736a8 P a8202c736da24a6eb4b0bd6c682b3557 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:28.665889 21905 raft_consensus.cc:374] T 86c3df6f408f4947a444e697c31736a8 P 0205d6e78d5d4a2194e1b8ce27560aa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:28.665953 21905 raft_consensus.cc:729] T 86c3df6f408f4947a444e697c31736a8 P 0205d6e78d5d4a2194e1b8ce27560aa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0205d6e78d5d4a2194e1b8ce27560aa9, State: Initialized, Role: FOLLOWER
I20260808 02:22:28.665953 21906 raft_consensus.cc:729] T 86c3df6f408f4947a444e697c31736a8 P a8202c736da24a6eb4b0bd6c682b3557 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8202c736da24a6eb4b0bd6c682b3557, State: Initialized, Role: FOLLOWER
I20260808 02:22:28.666520 21906 consensus_queue.cc:260] T 86c3df6f408f4947a444e697c31736a8 P a8202c736da24a6eb4b0bd6c682b3557 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0205d6e78d5d4a2194e1b8ce27560aa9" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 39341 } } peers { permanent_uuid: "a8202c736da24a6eb4b0bd6c682b3557" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 33433 } } peers { permanent_uuid: "979b94eebb6a4b82b378cae65130da43" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 39413 } }
I20260808 02:22:28.666520 21905 consensus_queue.cc:260] T 86c3df6f408f4947a444e697c31736a8 P 0205d6e78d5d4a2194e1b8ce27560aa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0205d6e78d5d4a2194e1b8ce27560aa9" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 39341 } } peers { permanent_uuid: "a8202c736da24a6eb4b0bd6c682b3557" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 33433 } } peers { permanent_uuid: "979b94eebb6a4b82b378cae65130da43" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 39413 } }
I20260808 02:22:28.667927 21905 ts_tablet_manager.cc:1428] T 86c3df6f408f4947a444e697c31736a8 P 0205d6e78d5d4a2194e1b8ce27560aa9: Time spent starting tablet: real 0.007s user 0.009s sys 0.000s
I20260808 02:22:28.667927 21906 ts_tablet_manager.cc:1428] T 86c3df6f408f4947a444e697c31736a8 P a8202c736da24a6eb4b0bd6c682b3557: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 02:22:28.668378 21474 heartbeater.cc:503] Master 127.24.79.254:38025 was elected leader, sending a full tablet report...
I20260808 02:22:28.668778 21681 heartbeater.cc:503] Master 127.24.79.254:38025 was elected leader, sending a full tablet report...
I20260808 02:22:28.668643 21907 raft_consensus.cc:348] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0205d6e78d5d4a2194e1b8ce27560aa9" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 39341 } } peers { permanent_uuid: "a8202c736da24a6eb4b0bd6c682b3557" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 33433 } } peers { permanent_uuid: "979b94eebb6a4b82b378cae65130da43" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 39413 } }
I20260808 02:22:28.668987 21907 raft_consensus.cc:374] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:28.669070 21907 raft_consensus.cc:729] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 979b94eebb6a4b82b378cae65130da43, State: Initialized, Role: FOLLOWER
I20260808 02:22:28.669690 21907 consensus_queue.cc:260] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0205d6e78d5d4a2194e1b8ce27560aa9" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 39341 } } peers { permanent_uuid: "a8202c736da24a6eb4b0bd6c682b3557" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 33433 } } peers { permanent_uuid: "979b94eebb6a4b82b378cae65130da43" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 39413 } }
I20260808 02:22:28.671339 21907 ts_tablet_manager.cc:1428] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260808 02:22:28.671793 21884 heartbeater.cc:503] Master 127.24.79.254:38025 was elected leader, sending a full tablet report...
I20260808 02:22:28.801383 21916 raft_consensus.cc:482] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:22:28.801640 21916 raft_consensus.cc:504] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0205d6e78d5d4a2194e1b8ce27560aa9" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 39341 } } peers { permanent_uuid: "a8202c736da24a6eb4b0bd6c682b3557" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 33433 } } peers { permanent_uuid: "979b94eebb6a4b82b378cae65130da43" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 39413 } }
I20260808 02:22:28.802977 21916 leader_election.cc:288] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0205d6e78d5d4a2194e1b8ce27560aa9 (127.24.79.193:39341), a8202c736da24a6eb4b0bd6c682b3557 (127.24.79.194:33433)
I20260808 02:22:28.807289 21886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:28.803279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.194:33433 (local address 127.24.79.195:44469)
0808 02:22:28.803445 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:28.803463 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:28.803576 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:28.803967 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:28.803971 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:28.803999 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:28.804141 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:28.804151 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.805031 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.805035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:28.805974 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:28.805985 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.806788 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.806792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:28.806895 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:28.806962 (+ 67us) client_negotiation.cc:769] Sending connection context
0808 02:22:28.807040 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 02:22:28.807126 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 02:22:28.807533 21902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:28.803389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:44469 (local address 127.24.79.194:33433)
0808 02:22:28.803519 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:28.803524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:28.803545 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:22:28.803748 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:28.803752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:28.803812 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:28.803906 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:28.804273 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.804896 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.806140 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.806654 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.806763 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:28.806807 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:28.807329 (+ 522us) server_negotiation.cc:299] Negotiation successful
0808 02:22:28.807423 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 02:22:28.807663 21901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:28.803582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.79.195:40265 (local address 127.24.79.193:39341)
0808 02:22:28.803737 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:28.803741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:28.803931 (+ 190us) server_negotiation.cc:407] Connection header received
0808 02:22:28.803997 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:28.804000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:28.804050 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:28.804221 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:22:28.804578 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.805201 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.806476 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.807031 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.807135 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:28.807249 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:28.807436 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 02:22:28.807530 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 02:22:28.808033 21918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:28.803276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:39341 (local address 127.24.79.195:40265)
0808 02:22:28.803819 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:28.803831 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:28.803921 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:28.804242 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:28.804245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:28.804264 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:22:28.804397 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:28.804404 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.805345 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.805348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:28.806331 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:28.806343 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.807151 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.807156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:28.807274 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:28.807300 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 02:22:28.807822 (+ 522us) client_negotiation.cc:240] Negotiation successful
0808 02:22:28.807905 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":179,"thread_start_us":160,"threads_started":1}
I20260808 02:22:28.808231 21612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86c3df6f408f4947a444e697c31736a8" candidate_uuid: "979b94eebb6a4b82b378cae65130da43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8202c736da24a6eb4b0bd6c682b3557" is_pre_election: true
I20260808 02:22:28.808602 21612 raft_consensus.cc:2454] T 86c3df6f408f4947a444e697c31736a8 P a8202c736da24a6eb4b0bd6c682b3557 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 979b94eebb6a4b82b378cae65130da43 in term 0.
I20260808 02:22:28.808840 21395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86c3df6f408f4947a444e697c31736a8" candidate_uuid: "979b94eebb6a4b82b378cae65130da43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0205d6e78d5d4a2194e1b8ce27560aa9" is_pre_election: true
I20260808 02:22:28.809204 21395 raft_consensus.cc:2454] T 86c3df6f408f4947a444e697c31736a8 P 0205d6e78d5d4a2194e1b8ce27560aa9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 979b94eebb6a4b82b378cae65130da43 in term 0.
I20260808 02:22:28.809247 21710 leader_election.cc:302] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 979b94eebb6a4b82b378cae65130da43, a8202c736da24a6eb4b0bd6c682b3557; no voters:
I20260808 02:22:28.809662 21916 raft_consensus.cc:2781] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:22:28.809780 21916 raft_consensus.cc:482] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:22:28.809830 21916 raft_consensus.cc:3037] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:28.811070 21916 raft_consensus.cc:504] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0205d6e78d5d4a2194e1b8ce27560aa9" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 39341 } } peers { permanent_uuid: "a8202c736da24a6eb4b0bd6c682b3557" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 33433 } } peers { permanent_uuid: "979b94eebb6a4b82b378cae65130da43" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 39413 } }
I20260808 02:22:28.811457 21916 leader_election.cc:288] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43 [CANDIDATE]: Term 1 election: Requested vote from peers 0205d6e78d5d4a2194e1b8ce27560aa9 (127.24.79.193:39341), a8202c736da24a6eb4b0bd6c682b3557 (127.24.79.194:33433)
I20260808 02:22:28.811969 21612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86c3df6f408f4947a444e697c31736a8" candidate_uuid: "979b94eebb6a4b82b378cae65130da43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8202c736da24a6eb4b0bd6c682b3557"
I20260808 02:22:28.811969 21395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86c3df6f408f4947a444e697c31736a8" candidate_uuid: "979b94eebb6a4b82b378cae65130da43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0205d6e78d5d4a2194e1b8ce27560aa9"
I20260808 02:22:28.812127 21395 raft_consensus.cc:3037] T 86c3df6f408f4947a444e697c31736a8 P 0205d6e78d5d4a2194e1b8ce27560aa9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:28.812127 21612 raft_consensus.cc:3037] T 86c3df6f408f4947a444e697c31736a8 P a8202c736da24a6eb4b0bd6c682b3557 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:28.813593 21395 raft_consensus.cc:2454] T 86c3df6f408f4947a444e697c31736a8 P 0205d6e78d5d4a2194e1b8ce27560aa9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 979b94eebb6a4b82b378cae65130da43 in term 1.
I20260808 02:22:28.813593 21612 raft_consensus.cc:2454] T 86c3df6f408f4947a444e697c31736a8 P a8202c736da24a6eb4b0bd6c682b3557 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 979b94eebb6a4b82b378cae65130da43 in term 1.
I20260808 02:22:28.814087 21713 leader_election.cc:302] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0205d6e78d5d4a2194e1b8ce27560aa9, 979b94eebb6a4b82b378cae65130da43; no voters:
I20260808 02:22:28.814342 21916 raft_consensus.cc:2781] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:28.814630 21916 raft_consensus.cc:686] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43 [term 1 LEADER]: Becoming Leader. State: Replica: 979b94eebb6a4b82b378cae65130da43, State: Running, Role: LEADER
I20260808 02:22:28.815078 21916 consensus_queue.cc:237] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0205d6e78d5d4a2194e1b8ce27560aa9" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 39341 } } peers { permanent_uuid: "a8202c736da24a6eb4b0bd6c682b3557" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 33433 } } peers { permanent_uuid: "979b94eebb6a4b82b378cae65130da43" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 39413 } }
I20260808 02:22:28.819123 21182 catalog_manager.cc:5495] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43 reported cstate change: term changed from 0 to 1, leader changed from <none> to 979b94eebb6a4b82b378cae65130da43 (127.24.79.195). New cstate: current_term: 1 leader_uuid: "979b94eebb6a4b82b378cae65130da43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0205d6e78d5d4a2194e1b8ce27560aa9" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 39341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8202c736da24a6eb4b0bd6c682b3557" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 33433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "979b94eebb6a4b82b378cae65130da43" member_type: VOTER last_known_addr { host: "127.24.79.195" port: 39413 } health_report { overall_health: HEALTHY } } }
W20260808 02:22:28.827850 21885 tablet.cc:2369] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:22:28.842041 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:28.838068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53614 (local address 127.24.79.195:39413)
0808 02:22:28.838285 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:28.838290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:28.838306 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:22:28.838410 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:28.838412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:28.838472 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:28.838580 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:22:28.838872 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.839439 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.840430 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:28.840660 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:28.840758 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:28.841109 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:22:28.841584 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:22:28.841708 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:28.841798 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 02:22:28.841863 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 02:22:28.849074 21612 raft_consensus.cc:1261] T 86c3df6f408f4947a444e697c31736a8 P a8202c736da24a6eb4b0bd6c682b3557 [term 1 FOLLOWER]: Refusing update from remote peer 979b94eebb6a4b82b378cae65130da43: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:22:28.849074 21395 raft_consensus.cc:1261] T 86c3df6f408f4947a444e697c31736a8 P 0205d6e78d5d4a2194e1b8ce27560aa9 [term 1 FOLLOWER]: Refusing update from remote peer 979b94eebb6a4b82b378cae65130da43: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:22:28.850050 21919 consensus_queue.cc:1035] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0205d6e78d5d4a2194e1b8ce27560aa9" member_type: VOTER last_known_addr { host: "127.24.79.193" port: 39341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:22:28.850292 21916 consensus_queue.cc:1035] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8202c736da24a6eb4b0bd6c682b3557" member_type: VOTER last_known_addr { host: "127.24.79.194" port: 33433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:22:28.870427 21182 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:22:28.870639 21182 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:22:28.874086 21182 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a209582d179489a816996c393c237b4: Sending DeleteTablet for 3 replicas of tablet 86c3df6f408f4947a444e697c31736a8
I20260808 02:22:28.875247 21363 tablet_service.cc:1514] Processing DeleteTablet for tablet 86c3df6f408f4947a444e697c31736a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:22:28 UTC) from {username='slave'} at 127.0.0.1:41032
I20260808 02:22:28.875190 21574 tablet_service.cc:1514] Processing DeleteTablet for tablet 86c3df6f408f4947a444e697c31736a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:22:28 UTC) from {username='slave'} at 127.0.0.1:37110
I20260808 02:22:28.875425 21789 tablet_service.cc:1514] Processing DeleteTablet for tablet 86c3df6f408f4947a444e697c31736a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:22:28 UTC) from {username='slave'} at 127.0.0.1:53602
I20260808 02:22:28.876065 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 21278
I20260808 02:22:28.875979 21944 tablet_replica.cc:330] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43: stopping tablet replica
I20260808 02:22:28.886758 21944 raft_consensus.cc:2229] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:22:28.887300 21944 raft_consensus.cc:2258] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:22:28.888623 21942 tablet_replica.cc:330] T 86c3df6f408f4947a444e697c31736a8 P a8202c736da24a6eb4b0bd6c682b3557: stopping tablet replica
I20260808 02:22:28.888975 21944 ts_tablet_manager.cc:1905] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:22:28.889012 21942 raft_consensus.cc:2229] T 86c3df6f408f4947a444e697c31736a8 P a8202c736da24a6eb4b0bd6c682b3557 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:22:28.889417 21942 raft_consensus.cc:2258] T 86c3df6f408f4947a444e697c31736a8 P a8202c736da24a6eb4b0bd6c682b3557 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:22:28.891595 21942 ts_tablet_manager.cc:1905] T 86c3df6f408f4947a444e697c31736a8 P a8202c736da24a6eb4b0bd6c682b3557: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:22:28.892338 21944 ts_tablet_manager.cc:1918] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:22:28.892481 21944 log.cc:1196] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-2/wal/wals/86c3df6f408f4947a444e697c31736a8
I20260808 02:22:28.892854 21944 ts_tablet_manager.cc:1939] T 86c3df6f408f4947a444e697c31736a8 P 979b94eebb6a4b82b378cae65130da43: Deleting consensus metadata
I20260808 02:22:28.894455 21171 catalog_manager.cc:4841] TS 979b94eebb6a4b82b378cae65130da43 (127.24.79.195:39413): tablet 86c3df6f408f4947a444e697c31736a8 (table test-table [id=a7379b2603844adc8ed4506b557f5a53]) successfully deleted
I20260808 02:22:28.895231 21942 ts_tablet_manager.cc:1918] T 86c3df6f408f4947a444e697c31736a8 P a8202c736da24a6eb4b0bd6c682b3557: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:22:28.895349 21942 log.cc:1196] T 86c3df6f408f4947a444e697c31736a8 P a8202c736da24a6eb4b0bd6c682b3557: Deleting WAL directory at /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155464663577-24895-0/minicluster-data/ts-1/wal/wals/86c3df6f408f4947a444e697c31736a8
I20260808 02:22:28.895644 21942 ts_tablet_manager.cc:1939] T 86c3df6f408f4947a444e697c31736a8 P a8202c736da24a6eb4b0bd6c682b3557: Deleting consensus metadata
I20260808 02:22:28.897198 21168 catalog_manager.cc:4841] TS a8202c736da24a6eb4b0bd6c682b3557 (127.24.79.194:33433): tablet 86c3df6f408f4947a444e697c31736a8 (table test-table [id=a7379b2603844adc8ed4506b557f5a53]) successfully deleted
W20260808 02:22:28.897850 21171 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.79.193:39341 (error 108)
I20260808 02:22:28.899040 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 21478
I20260808 02:22:28.899259 21900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:28.898888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.193:39341 (local address 127.0.0.1:41042)
0808 02:22:28.899074 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:28.899158 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.193:39341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260808 02:22:28.915992 21171 catalog_manager.cc:4584] TS 0205d6e78d5d4a2194e1b8ce27560aa9 (127.24.79.193:39341): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 86c3df6f408f4947a444e697c31736a8: Network error: Client connection negotiation failed: client connection to 127.24.79.193:39341: connect: Connection refused (error 111)
I20260808 02:22:28.924054 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 21685
I20260808 02:22:28.938709 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 21146
2026-08-08T02:22:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1345 ms)
[----------] 6 tests from EncryptionPolicyTest (12208 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:22:29 dist-test-slave-1vmg krb5kdc[21948](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:22:29 dist-test-slave-1vmg krb5kdc[21948](info): set up 2 sockets
Aug 08 02:22:29 dist-test-slave-1vmg krb5kdc[21948](info): commencing operation
krb5kdc: starting...
W20260808 02:22:30.115803 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:22:30 dist-test-slave-1vmg krb5kdc[21948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:22:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:22:30.271947 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35435
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:30.417516 21968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:30.417896 21968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:30.417963 21968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:30.422356 21968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:30.422468 21968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:30.422500 21968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:30.422524 21968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:30.422545 21968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:30.428556 21968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35435
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:30.429913 21968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:30.431144 21968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:30.441251 21977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:30.441330 21968 server_base.cc:1034] running on GCE node
W20260808 02:22:30.441330 21980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:30.441285 21976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:30.442346 21968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:30.443778 21968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:30.445026 21968 hybrid_clock.cc:648] HybridClock initialized: now 1786155750444995 us; error 50 us; skew 500 ppm
Aug 08 02:22:30 dist-test-slave-1vmg krb5kdc[21948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155750, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:30.448516 21968 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:22:30.449759 21968 webserver.cc:460] Webserver started at http://127.0.0.1:38149/ using document root <none> and password file <none>
I20260808 02:22:30.450383 21968 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:30.450455 21968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:30.450695 21968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:30.453106 21968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "5f2703b10f5e4d57877dad5bfbea0216"
format_stamp: "Formatted at 2026-08-08 02:22:30 on dist-test-slave-1vmg"
server_key: "784c923254f9cd338df4a4486cdacefa"
server_key_iv: "5fc61d19dd1f24d7e29ee1b01d3f68a1"
server_key_version: "encryptionkey@0"
I20260808 02:22:30.453783 21968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "5f2703b10f5e4d57877dad5bfbea0216"
format_stamp: "Formatted at 2026-08-08 02:22:30 on dist-test-slave-1vmg"
server_key: "784c923254f9cd338df4a4486cdacefa"
server_key_iv: "5fc61d19dd1f24d7e29ee1b01d3f68a1"
server_key_version: "encryptionkey@0"
I20260808 02:22:30.458258 21968 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 02:22:30.461452 21992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:30.463205 21968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:22:30.463356 21968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "5f2703b10f5e4d57877dad5bfbea0216"
format_stamp: "Formatted at 2026-08-08 02:22:30 on dist-test-slave-1vmg"
server_key: "784c923254f9cd338df4a4486cdacefa"
server_key_iv: "5fc61d19dd1f24d7e29ee1b01d3f68a1"
server_key_version: "encryptionkey@0"
I20260808 02:22:30.463500 21968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:30.474574 21968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:30.478286 21968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:30.478498 21968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:30.486353 21968 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35435
I20260808 02:22:30.486390 22074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35435 every 8 connection(s)
I20260808 02:22:30.487746 21968 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:30.491310 22075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:30.491991 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 21968
I20260808 02:22:30.492138 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:30.492443 24895 external_mini_cluster.cc:1428] Setting key 5266b8187ed3e719a7de8e6246f0e4d0
I20260808 02:22:30.498777 22075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f2703b10f5e4d57877dad5bfbea0216: Bootstrap starting.
I20260808 02:22:30.501988 22075 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f2703b10f5e4d57877dad5bfbea0216: Neither blocks nor log segments found. Creating new log.
Aug 08 02:22:30 dist-test-slave-1vmg krb5kdc[21948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 02:22:30.502940 22075 log.cc:824] T 00000000000000000000000000000000 P 5f2703b10f5e4d57877dad5bfbea0216: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:30.505888 22075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f2703b10f5e4d57877dad5bfbea0216: No bootstrap required, opened a new log
I20260808 02:22:30.509662 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:30.494077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37551 (local address 127.0.0.1:35435)
0808 02:22:30.494563 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:30.494582 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:30.494626 (+ 44us) server_negotiation.cc:407] Connection header received
0808 02:22:30.495392 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:30.495415 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:30.495820 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:30.496136 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:30.497209 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:30.498256 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:30.499495 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:30.500035 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:30.500161 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:30.503412 (+ 3251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:30.503420 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:30.503444 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:30.503484 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:30.505738 (+ 2254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:30.506448 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:30.506455 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:30.506462 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:30.506647 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:30.507045 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:30.507048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:30.507050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:30.507643 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:30.507880 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:30.508251 (+ 371us) server_negotiation.cc:299] Negotiation successful
0808 02:22:30.508546 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":125,"threads_started":1}
I20260808 02:22:30.510776 22075 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f2703b10f5e4d57877dad5bfbea0216 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f2703b10f5e4d57877dad5bfbea0216" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35435 } }
I20260808 02:22:30.511093 22075 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f2703b10f5e4d57877dad5bfbea0216 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:30.511139 22075 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f2703b10f5e4d57877dad5bfbea0216 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f2703b10f5e4d57877dad5bfbea0216, State: Initialized, Role: FOLLOWER
I20260808 02:22:30.511616 22075 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f2703b10f5e4d57877dad5bfbea0216 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f2703b10f5e4d57877dad5bfbea0216" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35435 } }
I20260808 02:22:30.511763 22075 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f2703b10f5e4d57877dad5bfbea0216 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:30.511837 22075 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f2703b10f5e4d57877dad5bfbea0216 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:30.511943 22075 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f2703b10f5e4d57877dad5bfbea0216 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:30.513139 22075 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f2703b10f5e4d57877dad5bfbea0216 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f2703b10f5e4d57877dad5bfbea0216" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35435 } }
I20260808 02:22:30.513372 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:22:30.513571 22075 leader_election.cc:302] T 00000000000000000000000000000000 P 5f2703b10f5e4d57877dad5bfbea0216 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f2703b10f5e4d57877dad5bfbea0216; no voters:
I20260808 02:22:30.514075 22075 leader_election.cc:288] T 00000000000000000000000000000000 P 5f2703b10f5e4d57877dad5bfbea0216 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:22:30.514242 22080 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f2703b10f5e4d57877dad5bfbea0216 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:30.514442 22080 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f2703b10f5e4d57877dad5bfbea0216 [term 1 LEADER]: Becoming Leader. State: Replica: 5f2703b10f5e4d57877dad5bfbea0216, State: Running, Role: LEADER
I20260808 02:22:30.514822 22080 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f2703b10f5e4d57877dad5bfbea0216 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f2703b10f5e4d57877dad5bfbea0216" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35435 } }
I20260808 02:22:30.515712 22075 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f2703b10f5e4d57877dad5bfbea0216 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:30.516832 22083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f2703b10f5e4d57877dad5bfbea0216 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f2703b10f5e4d57877dad5bfbea0216" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f2703b10f5e4d57877dad5bfbea0216" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35435 } } }
I20260808 02:22:30.517001 22083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f2703b10f5e4d57877dad5bfbea0216 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:30.517343 22083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f2703b10f5e4d57877dad5bfbea0216 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f2703b10f5e4d57877dad5bfbea0216. Latest consensus state: current_term: 1 leader_uuid: "5f2703b10f5e4d57877dad5bfbea0216" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f2703b10f5e4d57877dad5bfbea0216" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35435 } } }
I20260808 02:22:30.517457 22083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f2703b10f5e4d57877dad5bfbea0216 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:30.517935 22103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:22:30.521801 22103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:22:30.526237 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:30.515884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42069 (local address 127.0.0.1:35435)
0808 02:22:30.516015 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:30.516019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:30.516369 (+ 350us) server_negotiation.cc:407] Connection header received
0808 02:22:30.516581 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:30.516585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:30.516635 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:30.516690 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:30.517979 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:30.518797 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:30.520137 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:30.520525 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:30.520627 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:30.522454 (+ 1827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:30.522459 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:30.522462 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:30.522498 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:30.524071 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:30.524651 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:30.524655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:30.524657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:30.524752 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:30.525092 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:30.525096 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:30.525098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:30.525310 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:30.525451 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:30.525831 (+ 380us) server_negotiation.cc:299] Negotiation successful
0808 02:22:30.525980 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 02:22:30.529734 22103 catalog_manager.cc:1347] Generated new cluster ID: 1fd25859291d464f86445175fdf388c0
I20260808 02:22:30.529834 22103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:22:30.544785 22103 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:22:30.546159 22103 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:22:30.555714 22103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f2703b10f5e4d57877dad5bfbea0216: Generated new TSK 0
I20260808 02:22:30.556596 22103 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:22:30.566126 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 21968
W20260808 02:22:30.582787 21946 connection.cc:535] client connection to 127.0.0.1:35435 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35435): Connection reset by peer (error 104)
2026-08-08T02:22:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1625 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:22:30 dist-test-slave-1vmg krb5kdc[22123](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:22:30 dist-test-slave-1vmg krb5kdc[22123](info): set up 2 sockets
Aug 08 02:22:30 dist-test-slave-1vmg krb5kdc[22123](info): commencing operation
krb5kdc: starting...
W20260808 02:22:31.732762 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:22:31 dist-test-slave-1vmg krb5kdc[22123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:22:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:22:31.895752 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35561
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35561
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:32.032732 22149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:32.033130 22149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:32.033201 22149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:32.038264 22149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:32.038380 22149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:32.038409 22149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:32.038429 22149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:32.038450 22149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:32.044546 22149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35561
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:32.045847 22149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:32.047081 22149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:32.058151 22161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:32.058193 22158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:32.058128 22156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:32.058414 22149 server_base.cc:1034] running on GCE node
I20260808 02:22:32.059231 22149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:32.060981 22149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:32.062213 22149 hybrid_clock.cc:648] HybridClock initialized: now 1786155752062192 us; error 42 us; skew 500 ppm
Aug 08 02:22:32 dist-test-slave-1vmg krb5kdc[22123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155752, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:32.065487 22149 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:22:32.066598 22149 webserver.cc:460] Webserver started at http://127.0.0.1:37789/ using document root <none> and password file <none>
I20260808 02:22:32.067277 22149 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:32.067353 22149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:32.067569 22149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:32.069998 22149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "45b21a6353684d8d833cb71a87460825"
format_stamp: "Formatted at 2026-08-08 02:22:32 on dist-test-slave-1vmg"
server_key: "68085160cccafe95d3e9ffab3de8aac8"
server_key_iv: "8e3175801a09560fe7967b5d0038d150"
server_key_version: "encryptionkey@0"
I20260808 02:22:32.070713 22149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "45b21a6353684d8d833cb71a87460825"
format_stamp: "Formatted at 2026-08-08 02:22:32 on dist-test-slave-1vmg"
server_key: "68085160cccafe95d3e9ffab3de8aac8"
server_key_iv: "8e3175801a09560fe7967b5d0038d150"
server_key_version: "encryptionkey@0"
I20260808 02:22:32.075542 22149 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260808 02:22:32.078814 22170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:32.080806 22149 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:22:32.081003 22149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "45b21a6353684d8d833cb71a87460825"
format_stamp: "Formatted at 2026-08-08 02:22:32 on dist-test-slave-1vmg"
server_key: "68085160cccafe95d3e9ffab3de8aac8"
server_key_iv: "8e3175801a09560fe7967b5d0038d150"
server_key_version: "encryptionkey@0"
I20260808 02:22:32.081167 22149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:32.116233 22149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:32.120534 22149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:32.120780 22149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:32.128917 22149 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35561
I20260808 02:22:32.128957 22253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35561 every 8 connection(s)
I20260808 02:22:32.130053 22149 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:32.133749 22256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:32.136631 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 22149
I20260808 02:22:32.136775 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:32.137079 24895 external_mini_cluster.cc:1428] Setting key 42227b4ae6e0d4bff9c3d58117c280e2
I20260808 02:22:32.141619 22256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45b21a6353684d8d833cb71a87460825: Bootstrap starting.
I20260808 02:22:32.144804 22256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45b21a6353684d8d833cb71a87460825: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:32.145895 22256 log.cc:824] T 00000000000000000000000000000000 P 45b21a6353684d8d833cb71a87460825: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:32.148809 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:32.138663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51257 (local address 127.0.0.1:35561)
0808 02:22:32.139233 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:32.139258 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:32.139310 (+ 52us) server_negotiation.cc:407] Connection header received
0808 02:22:32.140248 (+ 938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:32.140280 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:32.140813 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:32.141268 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:32.141770 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:32.143346 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:32.144410 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:32.145171 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:32.145369 (+ 198us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:32.146369 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:32.146376 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:32.146398 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:32.146886 (+ 488us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:32.147004 (+ 118us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:32.147045 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:32.147370 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:32.147515 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 02:22:32.147722 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":182,"threads_started":1}
I20260808 02:22:32.148998 22256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45b21a6353684d8d833cb71a87460825: No bootstrap required, opened a new log
I20260808 02:22:32.152949 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:22:32.153777 22256 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45b21a6353684d8d833cb71a87460825 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45b21a6353684d8d833cb71a87460825" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35561 } }
I20260808 02:22:32.154064 22256 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45b21a6353684d8d833cb71a87460825 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:32.154138 22256 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45b21a6353684d8d833cb71a87460825 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45b21a6353684d8d833cb71a87460825, State: Initialized, Role: FOLLOWER
I20260808 02:22:32.154688 22256 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45b21a6353684d8d833cb71a87460825 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45b21a6353684d8d833cb71a87460825" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35561 } }
I20260808 02:22:32.154856 22256 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45b21a6353684d8d833cb71a87460825 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:32.155053 22256 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45b21a6353684d8d833cb71a87460825 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:32.155290 22256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45b21a6353684d8d833cb71a87460825 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:32.156785 22256 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45b21a6353684d8d833cb71a87460825 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45b21a6353684d8d833cb71a87460825" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35561 } }
I20260808 02:22:32.157436 22256 leader_election.cc:302] T 00000000000000000000000000000000 P 45b21a6353684d8d833cb71a87460825 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45b21a6353684d8d833cb71a87460825; no voters:
I20260808 02:22:32.157953 22256 leader_election.cc:288] T 00000000000000000000000000000000 P 45b21a6353684d8d833cb71a87460825 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:22:32.158164 22271 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45b21a6353684d8d833cb71a87460825 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:32.158445 22271 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45b21a6353684d8d833cb71a87460825 [term 1 LEADER]: Becoming Leader. State: Replica: 45b21a6353684d8d833cb71a87460825, State: Running, Role: LEADER
I20260808 02:22:32.158844 22271 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45b21a6353684d8d833cb71a87460825 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45b21a6353684d8d833cb71a87460825" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35561 } }
I20260808 02:22:32.159540 22256 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45b21a6353684d8d833cb71a87460825 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:32.160948 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:32.155478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49175 (local address 127.0.0.1:35561)
0808 02:22:32.155627 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:32.155632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:32.155998 (+ 366us) server_negotiation.cc:407] Connection header received
0808 02:22:32.156123 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:32.156125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:32.156174 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:32.156342 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:32.156745 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:32.157333 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:32.158557 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:32.158889 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:32.159009 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:32.160313 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:32.160318 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:32.160321 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:32.160338 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:32.160373 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:32.160387 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:32.160454 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:32.160742 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 02:22:32.160793 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 02:22:32.160881 22274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45b21a6353684d8d833cb71a87460825 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45b21a6353684d8d833cb71a87460825" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45b21a6353684d8d833cb71a87460825" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35561 } } }
I20260808 02:22:32.161022 22274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45b21a6353684d8d833cb71a87460825 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:32.161563 22275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45b21a6353684d8d833cb71a87460825 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45b21a6353684d8d833cb71a87460825. Latest consensus state: current_term: 1 leader_uuid: "45b21a6353684d8d833cb71a87460825" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45b21a6353684d8d833cb71a87460825" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35561 } } }
I20260808 02:22:32.161746 22275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45b21a6353684d8d833cb71a87460825 [sys.catalog]: This master's current role is: LEADER
W20260808 02:22:32.165840 22290 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 45b21a6353684d8d833cb71a87460825: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:22:32.166208 22290 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:22:32.166317 22284 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:22:32.167276 22284 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:22:32.174190 22284 catalog_manager.cc:1347] Generated new cluster ID: fcfeb6673829459da92746b5262d8799
I20260808 02:22:32.174297 22284 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:22:32.208565 22284 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:22:32.209683 22284 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:22:32.215015 22284 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45b21a6353684d8d833cb71a87460825: Generated new TSK 0
I20260808 02:22:32.215981 22284 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:22:32.232645 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 22149
W20260808 02:22:32.252594 22121 connection.cc:535] client connection to 127.0.0.1:35561 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35561): Connection reset by peer (error 104)
2026-08-08T02:22:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1671 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:22:32 dist-test-slave-1vmg krb5kdc[22299](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:22:32 dist-test-slave-1vmg krb5kdc[22299](info): set up 2 sockets
Aug 08 02:22:32 dist-test-slave-1vmg krb5kdc[22299](info): commencing operation
krb5kdc: starting...
W20260808 02:22:33.427340 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:22:33 dist-test-slave-1vmg krb5kdc[22299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:22:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:22:33.584576 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42777
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:33.711668 22319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:33.712162 22319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:33.712270 22319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:33.718624 22319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:33.718746 22319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:33.718778 22319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:33.718806 22319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:33.718829 22319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:33.727680 22319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42777
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:33.729377 22319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:33.731009 22319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:33.742137 22330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:33.742141 22324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:33.742141 22327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:33.743709 22319 server_base.cc:1034] running on GCE node
I20260808 02:22:33.744614 22319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:33.746347 22319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:33.747603 22319 hybrid_clock.cc:648] HybridClock initialized: now 1786155753747521 us; error 108 us; skew 500 ppm
Aug 08 02:22:33 dist-test-slave-1vmg krb5kdc[22299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155753, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:33.751767 22319 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:22:33.753153 22319 webserver.cc:460] Webserver started at http://127.0.0.1:45805/ using document root <none> and password file <none>
I20260808 02:22:33.754061 22319 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:33.754165 22319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:33.754441 22319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:33.757319 22319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "7b717d5114c847649da694c903a58881"
format_stamp: "Formatted at 2026-08-08 02:22:33 on dist-test-slave-1vmg"
server_key: "62f9393fa48c7975a6513ca562c992d5"
server_key_iv: "7a692dba1901c67d30cb1859365fe8d2"
server_key_version: "encryptionkey@0"
I20260808 02:22:33.758065 22319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "7b717d5114c847649da694c903a58881"
format_stamp: "Formatted at 2026-08-08 02:22:33 on dist-test-slave-1vmg"
server_key: "62f9393fa48c7975a6513ca562c992d5"
server_key_iv: "7a692dba1901c67d30cb1859365fe8d2"
server_key_version: "encryptionkey@0"
I20260808 02:22:33.762818 22319 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:22:33.765954 22342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:33.767688 22319 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:22:33.767912 22319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "7b717d5114c847649da694c903a58881"
format_stamp: "Formatted at 2026-08-08 02:22:33 on dist-test-slave-1vmg"
server_key: "62f9393fa48c7975a6513ca562c992d5"
server_key_iv: "7a692dba1901c67d30cb1859365fe8d2"
server_key_version: "encryptionkey@0"
I20260808 02:22:33.768064 22319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:33.773202 22319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:33.776692 22319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:33.776902 22319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:33.785617 22319 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42777
I20260808 02:22:33.785681 22413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42777 every 8 connection(s)
I20260808 02:22:33.786782 22319 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:33.791564 22414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:33.794416 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 22319
I20260808 02:22:33.794621 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:33.795096 24895 external_mini_cluster.cc:1428] Setting key 48d313158ea6535f8c7b168f48e3b8ff
I20260808 02:22:33.801147 22414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b717d5114c847649da694c903a58881: Bootstrap starting.
I20260808 02:22:33.801651 22422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:33.796973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47923 (local address 127.0.0.1:42777)
0808 02:22:33.797567 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:33.797585 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:33.797633 (+ 48us) server_negotiation.cc:407] Connection header received
0808 02:22:33.798374 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:33.798396 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:33.798738 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:33.799339 (+ 601us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:33.800166 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:33.800172 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:33.800198 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:33.800210 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:33.800274 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:33.800296 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:33.800446 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:33.800529 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 02:22:33.800724 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":191,"threads_started":1}
I20260808 02:22:33.804462 22414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b717d5114c847649da694c903a58881: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:33.805164 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:22:33.805538 22414 log.cc:824] T 00000000000000000000000000000000 P 7b717d5114c847649da694c903a58881: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:33.808512 22414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b717d5114c847649da694c903a58881: No bootstrap required, opened a new log
I20260808 02:22:33.810315 22422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:33.808793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35403 (local address 127.0.0.1:42777)
0808 02:22:33.808919 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:33.808926 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:33.808940 (+ 14us) server_negotiation.cc:407] Connection header received
0808 02:22:33.808982 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:33.808985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:33.809016 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:33.809146 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:33.809888 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:33.809891 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:33.809893 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:33.809905 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:33.809933 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:33.809944 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:33.810021 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:33.810111 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 02:22:33.810170 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 02:22:33.813844 22414 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b717d5114c847649da694c903a58881 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b717d5114c847649da694c903a58881" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42777 } }
I20260808 02:22:33.814241 22414 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b717d5114c847649da694c903a58881 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:33.814299 22414 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b717d5114c847649da694c903a58881 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b717d5114c847649da694c903a58881, State: Initialized, Role: FOLLOWER
I20260808 02:22:33.814888 22414 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b717d5114c847649da694c903a58881 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b717d5114c847649da694c903a58881" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42777 } }
I20260808 02:22:33.815107 22414 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b717d5114c847649da694c903a58881 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:33.815183 22414 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b717d5114c847649da694c903a58881 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:33.815313 22414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b717d5114c847649da694c903a58881 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:33.816764 22414 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b717d5114c847649da694c903a58881 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b717d5114c847649da694c903a58881" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42777 } }
I20260808 02:22:33.817301 22414 leader_election.cc:302] T 00000000000000000000000000000000 P 7b717d5114c847649da694c903a58881 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b717d5114c847649da694c903a58881; no voters:
I20260808 02:22:33.817765 22414 leader_election.cc:288] T 00000000000000000000000000000000 P 7b717d5114c847649da694c903a58881 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:22:33.817930 22434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b717d5114c847649da694c903a58881 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:33.818178 22434 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b717d5114c847649da694c903a58881 [term 1 LEADER]: Becoming Leader. State: Replica: 7b717d5114c847649da694c903a58881, State: Running, Role: LEADER
I20260808 02:22:33.818665 22434 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b717d5114c847649da694c903a58881 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b717d5114c847649da694c903a58881" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42777 } }
I20260808 02:22:33.819460 22414 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b717d5114c847649da694c903a58881 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:33.820542 22436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b717d5114c847649da694c903a58881 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b717d5114c847649da694c903a58881" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b717d5114c847649da694c903a58881" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42777 } } }
I20260808 02:22:33.820708 22436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b717d5114c847649da694c903a58881 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:33.821115 22437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b717d5114c847649da694c903a58881 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b717d5114c847649da694c903a58881. Latest consensus state: current_term: 1 leader_uuid: "7b717d5114c847649da694c903a58881" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b717d5114c847649da694c903a58881" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42777 } } }
I20260808 02:22:33.821302 22437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b717d5114c847649da694c903a58881 [sys.catalog]: This master's current role is: LEADER
W20260808 02:22:33.825546 22457 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7b717d5114c847649da694c903a58881: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:22:33.825671 22457 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:22:33.825739 22451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:22:33.826692 22451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:22:33.834448 22451 catalog_manager.cc:1347] Generated new cluster ID: 2d49b57383a4456994c4e4f587483552
I20260808 02:22:33.834550 22451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:22:33.852851 22451 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:22:33.853240 22451 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:22:33.860780 22451 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b717d5114c847649da694c903a58881: Generated new TSK 0
I20260808 02:22:33.861867 22451 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:22:33.871730 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 22319
2026-08-08T02:22:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1630 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:22:33 dist-test-slave-1vmg krb5kdc[22466](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:22:33 dist-test-slave-1vmg krb5kdc[22466](info): set up 2 sockets
Aug 08 02:22:33 dist-test-slave-1vmg krb5kdc[22466](info): commencing operation
krb5kdc: starting...
W20260808 02:22:35.034852 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:22:35 dist-test-slave-1vmg krb5kdc[22466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:22:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:22:35.187203 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40663
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:35.313228 22489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:35.313534 22489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:35.313578 22489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:35.317790 22489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:35.317886 22489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:35.317905 22489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:35.317921 22489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:35.317935 22489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:35.324011 22489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:35.325205 22489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:35.326594 22489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:35.337957 22496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:35.338253 22499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:35.338091 22495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:35.338610 22489 server_base.cc:1034] running on GCE node
I20260808 02:22:35.339473 22489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:35.341184 22489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:35.342442 22489 hybrid_clock.cc:648] HybridClock initialized: now 1786155755342413 us; error 54 us; skew 500 ppm
Aug 08 02:22:35 dist-test-slave-1vmg krb5kdc[22466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155755, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:35.345894 22489 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:22:35.347187 22489 webserver.cc:460] Webserver started at http://127.0.0.1:33973/ using document root <none> and password file <none>
I20260808 02:22:35.347826 22489 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:35.347905 22489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:35.348114 22489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:35.350554 22489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "e5aba3658fd443498a75800bd2f2cd24"
format_stamp: "Formatted at 2026-08-08 02:22:35 on dist-test-slave-1vmg"
server_key: "f2d7c2188430998dd249250d0fc312e2"
server_key_iv: "95f5e8fc3f0e6697b5fcbf39d3bec1e6"
server_key_version: "encryptionkey@0"
I20260808 02:22:35.351428 22489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "e5aba3658fd443498a75800bd2f2cd24"
format_stamp: "Formatted at 2026-08-08 02:22:35 on dist-test-slave-1vmg"
server_key: "f2d7c2188430998dd249250d0fc312e2"
server_key_iv: "95f5e8fc3f0e6697b5fcbf39d3bec1e6"
server_key_version: "encryptionkey@0"
I20260808 02:22:35.356438 22489 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:22:35.359490 22514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:35.361160 22489 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:22:35.361375 22489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "e5aba3658fd443498a75800bd2f2cd24"
format_stamp: "Formatted at 2026-08-08 02:22:35 on dist-test-slave-1vmg"
server_key: "f2d7c2188430998dd249250d0fc312e2"
server_key_iv: "95f5e8fc3f0e6697b5fcbf39d3bec1e6"
server_key_version: "encryptionkey@0"
I20260808 02:22:35.361528 22489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:35.394110 22489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:35.398010 22489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:35.398236 22489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:35.406464 22489 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40663
I20260808 02:22:35.406544 22599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40663 every 8 connection(s)
I20260808 02:22:35.407600 22489 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:35.410702 22601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:35.417081 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 22489
I20260808 02:22:35.417232 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:35.417443 22601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5aba3658fd443498a75800bd2f2cd24: Bootstrap starting.
I20260808 02:22:35.417505 24895 external_mini_cluster.cc:1428] Setting key d8fde832ae1ab3a7f8630f2725e938c8
I20260808 02:22:35.420380 22601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5aba3658fd443498a75800bd2f2cd24: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:35.421388 22601 log.cc:824] T 00000000000000000000000000000000 P e5aba3658fd443498a75800bd2f2cd24: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:35.424175 22601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5aba3658fd443498a75800bd2f2cd24: No bootstrap required, opened a new log
Aug 08 02:22:35 dist-test-slave-1vmg krb5kdc[22466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 02:22:35.428748 22601 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5aba3658fd443498a75800bd2f2cd24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5aba3658fd443498a75800bd2f2cd24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40663 } }
I20260808 02:22:35.429059 22601 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5aba3658fd443498a75800bd2f2cd24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:35.429124 22601 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5aba3658fd443498a75800bd2f2cd24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5aba3658fd443498a75800bd2f2cd24, State: Initialized, Role: FOLLOWER
I20260808 02:22:35.429702 22601 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5aba3658fd443498a75800bd2f2cd24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5aba3658fd443498a75800bd2f2cd24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40663 } }
I20260808 02:22:35.429908 22601 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5aba3658fd443498a75800bd2f2cd24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:35.429991 22601 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5aba3658fd443498a75800bd2f2cd24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:35.430112 22601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5aba3658fd443498a75800bd2f2cd24 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:35.431210 22601 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5aba3658fd443498a75800bd2f2cd24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5aba3658fd443498a75800bd2f2cd24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40663 } }
I20260808 02:22:35.431648 22601 leader_election.cc:302] T 00000000000000000000000000000000 P e5aba3658fd443498a75800bd2f2cd24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5aba3658fd443498a75800bd2f2cd24; no voters:
I20260808 02:22:35.432087 22601 leader_election.cc:288] T 00000000000000000000000000000000 P e5aba3658fd443498a75800bd2f2cd24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:22:35.432271 22608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5aba3658fd443498a75800bd2f2cd24 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:35.432525 22608 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5aba3658fd443498a75800bd2f2cd24 [term 1 LEADER]: Becoming Leader. State: Replica: e5aba3658fd443498a75800bd2f2cd24, State: Running, Role: LEADER
I20260808 02:22:35.432912 22608 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5aba3658fd443498a75800bd2f2cd24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5aba3658fd443498a75800bd2f2cd24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40663 } }
I20260808 02:22:35.433560 22601 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5aba3658fd443498a75800bd2f2cd24 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:35.434809 22609 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5aba3658fd443498a75800bd2f2cd24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5aba3658fd443498a75800bd2f2cd24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5aba3658fd443498a75800bd2f2cd24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40663 } } }
I20260808 02:22:35.434998 22609 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5aba3658fd443498a75800bd2f2cd24 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:35.435377 22604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:35.418965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44833 (local address 127.0.0.1:40663)
0808 02:22:35.419509 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:35.419532 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:35.419581 (+ 49us) server_negotiation.cc:407] Connection header received
0808 02:22:35.420536 (+ 955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:35.420567 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:35.421068 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:35.421527 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:35.422458 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:35.424014 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:35.425066 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:35.425809 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:35.425996 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:35.429060 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:35.429069 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:35.429090 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:35.429135 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:35.431260 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:35.431904 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:35.431911 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:35.431917 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:35.432096 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:35.432479 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:35.432483 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:35.432485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:35.433120 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:35.433350 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:35.433749 (+ 399us) server_negotiation.cc:299] Negotiation successful
0808 02:22:35.434108 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":165,"threads_started":1}
I20260808 02:22:35.435667 22610 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5aba3658fd443498a75800bd2f2cd24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5aba3658fd443498a75800bd2f2cd24. Latest consensus state: current_term: 1 leader_uuid: "e5aba3658fd443498a75800bd2f2cd24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5aba3658fd443498a75800bd2f2cd24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40663 } } }
I20260808 02:22:35.436311 22610 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5aba3658fd443498a75800bd2f2cd24 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:35.439178 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 02:22:35.442137 22631 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e5aba3658fd443498a75800bd2f2cd24: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:22:35.442263 22631 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:22:35.442337 22622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:22:35.443660 22622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:22:35.450860 22604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:35.441866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46529 (local address 127.0.0.1:40663)
0808 02:22:35.441996 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:35.442001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:35.442190 (+ 189us) server_negotiation.cc:407] Connection header received
0808 02:22:35.442333 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:35.442336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:35.442389 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:35.442446 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:35.443575 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:35.444196 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:35.445044 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:35.445307 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:35.445402 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:35.447008 (+ 1606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:35.447013 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:35.447016 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:35.447052 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:35.448837 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:35.449436 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:35.449440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:35.449441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:35.449555 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:35.449921 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:35.449924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:35.449925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:35.450141 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:35.450290 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:35.450552 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 02:22:35.450693 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 02:22:35.451802 22622 catalog_manager.cc:1347] Generated new cluster ID: 38f683ba80c14109ab12a5d6435a38e9
I20260808 02:22:35.451872 22622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:22:35.481894 22622 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:22:35.483079 22622 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:22:35.501540 22622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5aba3658fd443498a75800bd2f2cd24: Generated new TSK 0
I20260808 02:22:35.502607 22622 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:22:35.515630 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 22489
2026-08-08T02:22:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1645 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:22:35 dist-test-slave-1vmg krb5kdc[22647](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:22:35 dist-test-slave-1vmg krb5kdc[22647](info): set up 2 sockets
Aug 08 02:22:35 dist-test-slave-1vmg krb5kdc[22647](info): commencing operation
krb5kdc: starting...
W20260808 02:22:36.681175 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:22:36 dist-test-slave-1vmg krb5kdc[22647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:22:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:22:36.827486 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44945
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:36.955217 22667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:36.955564 22667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:36.955631 22667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:36.959916 22667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:36.960011 22667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:36.960040 22667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:36.960062 22667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:36.960084 22667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:36.965976 22667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:36.967322 22667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:36.968493 22667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:36.978286 22673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:36.978276 22679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:36.978346 22667 server_base.cc:1034] running on GCE node
W20260808 02:22:36.978291 22675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:36.979357 22667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:36.980698 22667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:36.981909 22667 hybrid_clock.cc:648] HybridClock initialized: now 1786155756981887 us; error 45 us; skew 500 ppm
Aug 08 02:22:36 dist-test-slave-1vmg krb5kdc[22647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155756, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:36.985359 22667 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:22:36.986523 22667 webserver.cc:460] Webserver started at http://127.0.0.1:39193/ using document root <none> and password file <none>
I20260808 02:22:36.987221 22667 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:36.987310 22667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:36.987553 22667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:36.989923 22667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "04abc0e88d514f7ba2617cf5bbe66c3e"
format_stamp: "Formatted at 2026-08-08 02:22:36 on dist-test-slave-1vmg"
server_key: "caea7622cdbe911160af45119d075588"
server_key_iv: "8a6c65a7fc0a6d46f6a8ca3ecb834d64"
server_key_version: "encryptionkey@0"
I20260808 02:22:36.990602 22667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "04abc0e88d514f7ba2617cf5bbe66c3e"
format_stamp: "Formatted at 2026-08-08 02:22:36 on dist-test-slave-1vmg"
server_key: "caea7622cdbe911160af45119d075588"
server_key_iv: "8a6c65a7fc0a6d46f6a8ca3ecb834d64"
server_key_version: "encryptionkey@0"
I20260808 02:22:36.995198 22667 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 02:22:36.998133 22689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:36.999821 22667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:22:36.999984 22667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "04abc0e88d514f7ba2617cf5bbe66c3e"
format_stamp: "Formatted at 2026-08-08 02:22:36 on dist-test-slave-1vmg"
server_key: "caea7622cdbe911160af45119d075588"
server_key_iv: "8a6c65a7fc0a6d46f6a8ca3ecb834d64"
server_key_version: "encryptionkey@0"
I20260808 02:22:37.000123 22667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:37.016736 22667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:37.020848 22667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:37.021116 22667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:37.029858 22667 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44945
I20260808 02:22:37.029927 22771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44945 every 8 connection(s)
I20260808 02:22:37.031292 22667 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:37.034850 22772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:37.037228 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 22667
I20260808 02:22:37.037482 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:37.037837 24895 external_mini_cluster.cc:1428] Setting key e0c05c08e794bb3b4a856f3bb72d7fa2
I20260808 02:22:37.042374 22772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04abc0e88d514f7ba2617cf5bbe66c3e: Bootstrap starting.
I20260808 02:22:37.045055 22772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04abc0e88d514f7ba2617cf5bbe66c3e: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:37.046104 22772 log.cc:824] T 00000000000000000000000000000000 P 04abc0e88d514f7ba2617cf5bbe66c3e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:37.048169 22779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:37.039554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40661 (local address 127.0.0.1:44945)
0808 02:22:37.040254 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:37.040272 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:37.040315 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:22:37.041058 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:37.041081 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:37.041507 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:37.041880 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:37.042373 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:37.043492 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:37.044465 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:37.044987 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:37.045172 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:37.046143 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:37.046151 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:37.046171 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:37.046661 (+ 490us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:37.046730 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:37.046758 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:37.046856 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:37.047048 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 02:22:37.047304 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":288,"threads_started":1}
I20260808 02:22:37.048491 22772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04abc0e88d514f7ba2617cf5bbe66c3e: No bootstrap required, opened a new log
I20260808 02:22:37.051816 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:22:37.053045 22772 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04abc0e88d514f7ba2617cf5bbe66c3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04abc0e88d514f7ba2617cf5bbe66c3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44945 } }
I20260808 02:22:37.053484 22772 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04abc0e88d514f7ba2617cf5bbe66c3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:37.053571 22772 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04abc0e88d514f7ba2617cf5bbe66c3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04abc0e88d514f7ba2617cf5bbe66c3e, State: Initialized, Role: FOLLOWER
I20260808 02:22:37.054126 22772 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04abc0e88d514f7ba2617cf5bbe66c3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04abc0e88d514f7ba2617cf5bbe66c3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44945 } }
I20260808 02:22:37.054345 22772 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04abc0e88d514f7ba2617cf5bbe66c3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:37.054428 22772 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04abc0e88d514f7ba2617cf5bbe66c3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:37.054546 22772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04abc0e88d514f7ba2617cf5bbe66c3e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:37.056262 22772 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04abc0e88d514f7ba2617cf5bbe66c3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04abc0e88d514f7ba2617cf5bbe66c3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44945 } }
I20260808 02:22:37.056743 22772 leader_election.cc:302] T 00000000000000000000000000000000 P 04abc0e88d514f7ba2617cf5bbe66c3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04abc0e88d514f7ba2617cf5bbe66c3e; no voters:
I20260808 02:22:37.057209 22772 leader_election.cc:288] T 00000000000000000000000000000000 P 04abc0e88d514f7ba2617cf5bbe66c3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:22:37.057335 22788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04abc0e88d514f7ba2617cf5bbe66c3e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:37.057641 22788 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04abc0e88d514f7ba2617cf5bbe66c3e [term 1 LEADER]: Becoming Leader. State: Replica: 04abc0e88d514f7ba2617cf5bbe66c3e, State: Running, Role: LEADER
I20260808 02:22:37.058069 22788 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04abc0e88d514f7ba2617cf5bbe66c3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04abc0e88d514f7ba2617cf5bbe66c3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44945 } }
I20260808 02:22:37.058997 22772 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04abc0e88d514f7ba2617cf5bbe66c3e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:37.059170 22779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:37.054054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56075 (local address 127.0.0.1:44945)
0808 02:22:37.054222 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:37.054227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:37.054371 (+ 144us) server_negotiation.cc:407] Connection header received
0808 02:22:37.054573 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:37.054576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:37.054647 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:37.054748 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:37.055105 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:37.056013 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:37.057029 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:37.057416 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:37.057564 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:37.058568 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:37.058573 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:37.058576 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:37.058595 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:37.058631 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:37.058649 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:37.058743 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:37.058855 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 02:22:37.058957 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:22:37.060314 22791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04abc0e88d514f7ba2617cf5bbe66c3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04abc0e88d514f7ba2617cf5bbe66c3e. Latest consensus state: current_term: 1 leader_uuid: "04abc0e88d514f7ba2617cf5bbe66c3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04abc0e88d514f7ba2617cf5bbe66c3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44945 } } }
I20260808 02:22:37.060680 22790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04abc0e88d514f7ba2617cf5bbe66c3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04abc0e88d514f7ba2617cf5bbe66c3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04abc0e88d514f7ba2617cf5bbe66c3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44945 } } }
I20260808 02:22:37.060811 22791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04abc0e88d514f7ba2617cf5bbe66c3e [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:37.060844 22790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04abc0e88d514f7ba2617cf5bbe66c3e [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:37.061391 22802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:22:37.065172 22802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:22:37.073418 22802 catalog_manager.cc:1347] Generated new cluster ID: daccb983d7664150a6906fb847e769d2
I20260808 02:22:37.073580 22802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:22:37.086066 22802 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:22:37.088433 22802 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:22:37.098325 22802 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04abc0e88d514f7ba2617cf5bbe66c3e: Generated new TSK 0
I20260808 02:22:37.099318 22802 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:22:37.113466 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 22667
2026-08-08T02:22:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1599 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:22:37 dist-test-slave-1vmg krb5kdc[22819](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:22:37 dist-test-slave-1vmg krb5kdc[22819](info): set up 2 sockets
Aug 08 02:22:37 dist-test-slave-1vmg krb5kdc[22819](info): commencing operation
krb5kdc: starting...
W20260808 02:22:38.285670 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:22:38 dist-test-slave-1vmg krb5kdc[22819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:22:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:22:38.436075 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34621
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:38.559623 22837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:38.560027 22837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:38.560091 22837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:38.564435 22837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:38.564524 22837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:38.564551 22837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:38.564572 22837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:38.564595 22837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:38.570497 22837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:38.571856 22837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:38.572995 22837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:38.582840 22848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:38.582840 22851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:38.583069 22837 server_base.cc:1034] running on GCE node
W20260808 02:22:38.582908 22847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:38.583920 22837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:38.585256 22837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:38.586483 22837 hybrid_clock.cc:648] HybridClock initialized: now 1786155758586457 us; error 47 us; skew 500 ppm
Aug 08 02:22:38 dist-test-slave-1vmg krb5kdc[22819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155758, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:38.595153 22837 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:22:38.596416 22837 webserver.cc:460] Webserver started at http://127.0.0.1:40547/ using document root <none> and password file <none>
I20260808 02:22:38.597057 22837 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:38.597146 22837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:38.597353 22837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:38.599874 22837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "90e09c02455d4e73812dc295aeaa99bc"
format_stamp: "Formatted at 2026-08-08 02:22:38 on dist-test-slave-1vmg"
server_key: "0a5181d188232082e4abe46a6e590e5b"
server_key_iv: "c377e66bd9d3ce85f3f91651f2df6734"
server_key_version: "encryptionkey@0"
I20260808 02:22:38.600576 22837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "90e09c02455d4e73812dc295aeaa99bc"
format_stamp: "Formatted at 2026-08-08 02:22:38 on dist-test-slave-1vmg"
server_key: "0a5181d188232082e4abe46a6e590e5b"
server_key_iv: "c377e66bd9d3ce85f3f91651f2df6734"
server_key_version: "encryptionkey@0"
I20260808 02:22:38.605283 22837 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:22:38.608289 22863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:38.609880 22837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:22:38.610049 22837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "90e09c02455d4e73812dc295aeaa99bc"
format_stamp: "Formatted at 2026-08-08 02:22:38 on dist-test-slave-1vmg"
server_key: "0a5181d188232082e4abe46a6e590e5b"
server_key_iv: "c377e66bd9d3ce85f3f91651f2df6734"
server_key_version: "encryptionkey@0"
I20260808 02:22:38.610236 22837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:38.615806 22837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:38.619510 22837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:38.619736 22837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:38.627885 22837 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34621
I20260808 02:22:38.627919 22938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34621 every 8 connection(s)
I20260808 02:22:38.629030 22837 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:38.632320 22939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:38.635555 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 22837
I20260808 02:22:38.635735 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:38.636056 24895 external_mini_cluster.cc:1428] Setting key 207babfba2090aa8ce81ce4044732471
I20260808 02:22:38.639349 22939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90e09c02455d4e73812dc295aeaa99bc: Bootstrap starting.
I20260808 02:22:38.642098 22942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:38.637771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44467 (local address 127.0.0.1:34621)
0808 02:22:38.638290 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:38.638309 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:38.638349 (+ 40us) server_negotiation.cc:407] Connection header received
0808 02:22:38.639099 (+ 750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:38.639121 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:38.639448 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:38.639812 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:38.640744 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:38.640749 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:38.640764 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:38.640777 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:38.640843 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:38.640865 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:38.640991 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:38.641093 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 02:22:38.641312 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":172,"threads_started":1}
I20260808 02:22:38.642131 22939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90e09c02455d4e73812dc295aeaa99bc: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:38.643015 22939 log.cc:824] T 00000000000000000000000000000000 P 90e09c02455d4e73812dc295aeaa99bc: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:38.645157 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:22:38.645304 22939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90e09c02455d4e73812dc295aeaa99bc: No bootstrap required, opened a new log
I20260808 02:22:38.649678 22942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:38.647779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54455 (local address 127.0.0.1:34621)
0808 02:22:38.647921 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:38.647926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:38.648155 (+ 229us) server_negotiation.cc:407] Connection header received
0808 02:22:38.648275 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:38.648278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:38.648321 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:38.648418 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:38.649231 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:38.649235 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:38.649238 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:38.649251 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:38.649285 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:38.649298 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:38.649395 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:38.649486 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 02:22:38.649529 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 02:22:38.650126 22939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90e09c02455d4e73812dc295aeaa99bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e09c02455d4e73812dc295aeaa99bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34621 } }
I20260808 02:22:38.650413 22939 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90e09c02455d4e73812dc295aeaa99bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:38.650467 22939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90e09c02455d4e73812dc295aeaa99bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e09c02455d4e73812dc295aeaa99bc, State: Initialized, Role: FOLLOWER
I20260808 02:22:38.650965 22939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90e09c02455d4e73812dc295aeaa99bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e09c02455d4e73812dc295aeaa99bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34621 } }
I20260808 02:22:38.651124 22939 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90e09c02455d4e73812dc295aeaa99bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:38.651203 22939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90e09c02455d4e73812dc295aeaa99bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:38.651315 22939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90e09c02455d4e73812dc295aeaa99bc [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:38.652383 22939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90e09c02455d4e73812dc295aeaa99bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e09c02455d4e73812dc295aeaa99bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34621 } }
I20260808 02:22:38.652782 22939 leader_election.cc:302] T 00000000000000000000000000000000 P 90e09c02455d4e73812dc295aeaa99bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90e09c02455d4e73812dc295aeaa99bc; no voters:
I20260808 02:22:38.653209 22939 leader_election.cc:288] T 00000000000000000000000000000000 P 90e09c02455d4e73812dc295aeaa99bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:22:38.653329 22953 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90e09c02455d4e73812dc295aeaa99bc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:38.653571 22953 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90e09c02455d4e73812dc295aeaa99bc [term 1 LEADER]: Becoming Leader. State: Replica: 90e09c02455d4e73812dc295aeaa99bc, State: Running, Role: LEADER
I20260808 02:22:38.654001 22953 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90e09c02455d4e73812dc295aeaa99bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e09c02455d4e73812dc295aeaa99bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34621 } }
I20260808 02:22:38.654692 22939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90e09c02455d4e73812dc295aeaa99bc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:38.656069 22955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90e09c02455d4e73812dc295aeaa99bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90e09c02455d4e73812dc295aeaa99bc. Latest consensus state: current_term: 1 leader_uuid: "90e09c02455d4e73812dc295aeaa99bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e09c02455d4e73812dc295aeaa99bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34621 } } }
I20260808 02:22:38.656093 22954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90e09c02455d4e73812dc295aeaa99bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90e09c02455d4e73812dc295aeaa99bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e09c02455d4e73812dc295aeaa99bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34621 } } }
I20260808 02:22:38.656216 22955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90e09c02455d4e73812dc295aeaa99bc [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:38.656220 22954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90e09c02455d4e73812dc295aeaa99bc [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:38.656602 22969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:22:38.659802 22969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:22:38.666357 22969 catalog_manager.cc:1347] Generated new cluster ID: 42f43e1776b249f18588f7bf060aa5bc
I20260808 02:22:38.666473 22969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:22:38.677279 22969 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:22:38.677729 22969 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:22:38.683665 22969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90e09c02455d4e73812dc295aeaa99bc: Generated new TSK 0
I20260808 02:22:38.684576 22969 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:22:38.688809 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 22837
2026-08-08T02:22:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1577 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:22:38 dist-test-slave-1vmg krb5kdc[22986](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:22:38 dist-test-slave-1vmg krb5kdc[22986](info): set up 2 sockets
Aug 08 02:22:38 dist-test-slave-1vmg krb5kdc[22986](info): commencing operation
krb5kdc: starting...
W20260808 02:22:39.856745 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:22:39 dist-test-slave-1vmg krb5kdc[22986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:22:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:22:40.011765 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34169
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:40.144706 23011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:40.145087 23011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:40.145155 23011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:40.149824 23011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:40.149945 23011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:40.149976 23011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:40.150000 23011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:40.150019 23011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:40.156334 23011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:40.157691 23011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:40.159046 23011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:40.169432 23023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:40.169453 23018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:40.169504 23011 server_base.cc:1034] running on GCE node
W20260808 02:22:40.169456 23019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:40.170537 23011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:40.172089 23011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:40.173349 23011 hybrid_clock.cc:648] HybridClock initialized: now 1786155760173319 us; error 51 us; skew 500 ppm
Aug 08 02:22:40 dist-test-slave-1vmg krb5kdc[22986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155760, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:40.176679 23011 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:22:40.177919 23011 webserver.cc:460] Webserver started at http://127.0.0.1:36055/ using document root <none> and password file <none>
I20260808 02:22:40.178560 23011 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:40.178642 23011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:40.179060 23011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:40.181535 23011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "94b14a94ca7c49849120b0b9d21939a0"
format_stamp: "Formatted at 2026-08-08 02:22:40 on dist-test-slave-1vmg"
server_key: "e239b63bcab007e78ba54be6475c5d51"
server_key_iv: "1b2c1dcd10399aae530f7276b4d14817"
server_key_version: "encryptionkey@0"
I20260808 02:22:40.182164 23011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "94b14a94ca7c49849120b0b9d21939a0"
format_stamp: "Formatted at 2026-08-08 02:22:40 on dist-test-slave-1vmg"
server_key: "e239b63bcab007e78ba54be6475c5d51"
server_key_iv: "1b2c1dcd10399aae530f7276b4d14817"
server_key_version: "encryptionkey@0"
I20260808 02:22:40.186614 23011 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:22:40.189651 23036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:40.191355 23011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:22:40.191525 23011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "94b14a94ca7c49849120b0b9d21939a0"
format_stamp: "Formatted at 2026-08-08 02:22:40 on dist-test-slave-1vmg"
server_key: "e239b63bcab007e78ba54be6475c5d51"
server_key_iv: "1b2c1dcd10399aae530f7276b4d14817"
server_key_version: "encryptionkey@0"
I20260808 02:22:40.191669 23011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:40.206271 23011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:40.210592 23011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:40.210847 23011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:40.219013 23011 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34169
I20260808 02:22:40.219097 23098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34169 every 8 connection(s)
I20260808 02:22:40.220244 23011 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:40.222321 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 23011
I20260808 02:22:40.222470 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:40.222782 24895 external_mini_cluster.cc:1428] Setting key c8139c11e09a2dcda18f61cc6d76777b
I20260808 02:22:40.224010 23100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:40.231525 23100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94b14a94ca7c49849120b0b9d21939a0: Bootstrap starting.
Aug 08 02:22:40 dist-test-slave-1vmg krb5kdc[22986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 02:22:40.234658 23100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94b14a94ca7c49849120b0b9d21939a0: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:40.235608 23100 log.cc:824] T 00000000000000000000000000000000 P 94b14a94ca7c49849120b0b9d21939a0: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:40.238543 23100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94b14a94ca7c49849120b0b9d21939a0: No bootstrap required, opened a new log
I20260808 02:22:40.240756 23104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:40.224507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:57767 (local address 127.0.0.1:34169)
0808 02:22:40.225020 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:40.225042 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:40.225086 (+ 44us) server_negotiation.cc:407] Connection header received
0808 02:22:40.225781 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:40.225800 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:40.226188 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:40.226519 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:40.227584 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:40.228830 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:40.230170 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:40.230764 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:40.231038 (+ 274us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:40.234515 (+ 3477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:40.234522 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:40.234543 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:40.234584 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:40.236816 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:40.237577 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:40.237583 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:40.237590 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:40.237772 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:40.238210 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:40.238214 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:40.238215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:40.238750 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:40.238999 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:40.239361 (+ 362us) server_negotiation.cc:299] Negotiation successful
0808 02:22:40.239628 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":163,"threads_started":1}
I20260808 02:22:40.242316 23100 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94b14a94ca7c49849120b0b9d21939a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94b14a94ca7c49849120b0b9d21939a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34169 } }
I20260808 02:22:40.242650 23100 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94b14a94ca7c49849120b0b9d21939a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:40.242708 23100 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94b14a94ca7c49849120b0b9d21939a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94b14a94ca7c49849120b0b9d21939a0, State: Initialized, Role: FOLLOWER
I20260808 02:22:40.243217 23100 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94b14a94ca7c49849120b0b9d21939a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94b14a94ca7c49849120b0b9d21939a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34169 } }
I20260808 02:22:40.243399 23100 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94b14a94ca7c49849120b0b9d21939a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:40.243474 23100 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94b14a94ca7c49849120b0b9d21939a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:40.243588 23100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94b14a94ca7c49849120b0b9d21939a0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:40.244529 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:22:40.244972 23100 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94b14a94ca7c49849120b0b9d21939a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94b14a94ca7c49849120b0b9d21939a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34169 } }
I20260808 02:22:40.245397 23100 leader_election.cc:302] T 00000000000000000000000000000000 P 94b14a94ca7c49849120b0b9d21939a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94b14a94ca7c49849120b0b9d21939a0; no voters:
I20260808 02:22:40.245759 23100 leader_election.cc:288] T 00000000000000000000000000000000 P 94b14a94ca7c49849120b0b9d21939a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:22:40.245894 23108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94b14a94ca7c49849120b0b9d21939a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:40.246145 23108 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94b14a94ca7c49849120b0b9d21939a0 [term 1 LEADER]: Becoming Leader. State: Replica: 94b14a94ca7c49849120b0b9d21939a0, State: Running, Role: LEADER
I20260808 02:22:40.246565 23108 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94b14a94ca7c49849120b0b9d21939a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94b14a94ca7c49849120b0b9d21939a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34169 } }
I20260808 02:22:40.247447 23100 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94b14a94ca7c49849120b0b9d21939a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:40.248732 23117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94b14a94ca7c49849120b0b9d21939a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94b14a94ca7c49849120b0b9d21939a0. Latest consensus state: current_term: 1 leader_uuid: "94b14a94ca7c49849120b0b9d21939a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94b14a94ca7c49849120b0b9d21939a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34169 } } }
I20260808 02:22:40.248890 23117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94b14a94ca7c49849120b0b9d21939a0 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:40.249250 23116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94b14a94ca7c49849120b0b9d21939a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94b14a94ca7c49849120b0b9d21939a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94b14a94ca7c49849120b0b9d21939a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34169 } } }
I20260808 02:22:40.249362 23116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94b14a94ca7c49849120b0b9d21939a0 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:40.249828 23130 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:22:40.253659 23130 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:22:40.257440 23104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:40.246894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:48755 (local address 127.0.0.1:34169)
0808 02:22:40.247082 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:40.247087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:40.247349 (+ 262us) server_negotiation.cc:407] Connection header received
0808 02:22:40.247487 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:40.247490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:40.247544 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:40.247622 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:40.248669 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:40.249416 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:40.250679 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:40.250960 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:40.251064 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:40.253065 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:40.253069 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:40.253072 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:40.253108 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:40.255047 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:40.255835 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:40.255840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:40.255841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:40.255955 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:40.256378 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:40.256381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:40.256383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:40.256639 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:40.256810 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:40.257157 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 02:22:40.257277 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260808 02:22:40.261360 23130 catalog_manager.cc:1347] Generated new cluster ID: ef0ed56280e64bc58e8ff666775bf2d9
I20260808 02:22:40.261461 23130 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:22:40.279340 23130 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:22:40.280467 23130 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:22:40.291904 23130 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94b14a94ca7c49849120b0b9d21939a0: Generated new TSK 0
I20260808 02:22:40.292881 23130 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:22:40.308280 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 23011
W20260808 02:22:40.333585 22981 connection.cc:535] client connection to 127.0.0.1:34169 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34169): Connection reset by peer (error 104)
2026-08-08T02:22:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1626 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:22:40 dist-test-slave-1vmg krb5kdc[23144](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:22:40 dist-test-slave-1vmg krb5kdc[23144](info): set up 2 sockets
Aug 08 02:22:40 dist-test-slave-1vmg krb5kdc[23144](info): commencing operation
krb5kdc: starting...
W20260808 02:22:41.495168 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:22:41 dist-test-slave-1vmg krb5kdc[23144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:22:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:22:41.643172 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37697
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37697
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:41.770440 23163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:41.770778 23163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:41.770823 23163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:41.775136 23163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:41.775238 23163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:41.775259 23163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:41.775275 23163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:41.775292 23163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:41.781183 23163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37697
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:41.782449 23163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:41.783572 23163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:41.793218 23172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:41.793313 23171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:41.793294 23163 server_base.cc:1034] running on GCE node
W20260808 02:22:41.793188 23174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:41.794132 23163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:41.795468 23163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:41.796643 23163 hybrid_clock.cc:648] HybridClock initialized: now 1786155761796600 us; error 62 us; skew 500 ppm
Aug 08 02:22:41 dist-test-slave-1vmg krb5kdc[23144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155761, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:41.799808 23163 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:22:41.800928 23163 webserver.cc:460] Webserver started at http://127.0.0.1:37147/ using document root <none> and password file <none>
I20260808 02:22:41.801556 23163 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:41.801645 23163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:41.801884 23163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:41.804242 23163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "a18a2b9934b94aea9706586a7de4c9cd"
format_stamp: "Formatted at 2026-08-08 02:22:41 on dist-test-slave-1vmg"
server_key: "9afa2e75006ad74f4046a51cb6647fd4"
server_key_iv: "ab9c24e37d1062d6cdd18568a3900805"
server_key_version: "encryptionkey@0"
I20260808 02:22:41.804847 23163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "a18a2b9934b94aea9706586a7de4c9cd"
format_stamp: "Formatted at 2026-08-08 02:22:41 on dist-test-slave-1vmg"
server_key: "9afa2e75006ad74f4046a51cb6647fd4"
server_key_iv: "ab9c24e37d1062d6cdd18568a3900805"
server_key_version: "encryptionkey@0"
I20260808 02:22:41.809162 23163 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:22:41.812255 23189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:41.813952 23163 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:22:41.814154 23163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "a18a2b9934b94aea9706586a7de4c9cd"
format_stamp: "Formatted at 2026-08-08 02:22:41 on dist-test-slave-1vmg"
server_key: "9afa2e75006ad74f4046a51cb6647fd4"
server_key_iv: "ab9c24e37d1062d6cdd18568a3900805"
server_key_version: "encryptionkey@0"
I20260808 02:22:41.814292 23163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:41.826654 23163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:41.830679 23163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:41.830973 23163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:41.839398 23163 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37697
I20260808 02:22:41.839412 23266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37697 every 8 connection(s)
I20260808 02:22:41.840674 23163 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:41.842522 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 23163
I20260808 02:22:41.842677 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:41.843015 24895 external_mini_cluster.cc:1428] Setting key b0d0045f2a40fd656a6c8f369c4e55fe
I20260808 02:22:41.844254 23267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:41.851157 23267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a18a2b9934b94aea9706586a7de4c9cd: Bootstrap starting.
I20260808 02:22:41.853358 23275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:41.844623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:37459 (local address 127.0.0.1:37697)
0808 02:22:41.845181 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:41.845202 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:41.845246 (+ 44us) server_negotiation.cc:407] Connection header received
0808 02:22:41.845963 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:41.845983 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:41.846400 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:41.846815 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:41.847347 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:41.848443 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:41.849432 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:41.849996 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:41.850164 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:41.851228 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:41.851234 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:41.851252 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:41.851768 (+ 516us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:41.851846 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:41.851877 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:41.852015 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:41.852296 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 02:22:41.852473 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":176,"threads_started":1}
I20260808 02:22:41.853905 23267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a18a2b9934b94aea9706586a7de4c9cd: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:41.854743 23267 log.cc:824] T 00000000000000000000000000000000 P a18a2b9934b94aea9706586a7de4c9cd: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:41.857409 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:22:41.857681 23267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a18a2b9934b94aea9706586a7de4c9cd: No bootstrap required, opened a new log
I20260808 02:22:41.862212 23267 raft_consensus.cc:348] T 00000000000000000000000000000000 P a18a2b9934b94aea9706586a7de4c9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a18a2b9934b94aea9706586a7de4c9cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37697 } }
I20260808 02:22:41.862540 23267 raft_consensus.cc:374] T 00000000000000000000000000000000 P a18a2b9934b94aea9706586a7de4c9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:41.862593 23267 raft_consensus.cc:729] T 00000000000000000000000000000000 P a18a2b9934b94aea9706586a7de4c9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a18a2b9934b94aea9706586a7de4c9cd, State: Initialized, Role: FOLLOWER
I20260808 02:22:41.863214 23267 consensus_queue.cc:260] T 00000000000000000000000000000000 P a18a2b9934b94aea9706586a7de4c9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a18a2b9934b94aea9706586a7de4c9cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37697 } }
I20260808 02:22:41.863425 23267 raft_consensus.cc:388] T 00000000000000000000000000000000 P a18a2b9934b94aea9706586a7de4c9cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:41.863509 23267 raft_consensus.cc:482] T 00000000000000000000000000000000 P a18a2b9934b94aea9706586a7de4c9cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:41.863629 23267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a18a2b9934b94aea9706586a7de4c9cd [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:41.865015 23267 raft_consensus.cc:504] T 00000000000000000000000000000000 P a18a2b9934b94aea9706586a7de4c9cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a18a2b9934b94aea9706586a7de4c9cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37697 } }
I20260808 02:22:41.865432 23267 leader_election.cc:302] T 00000000000000000000000000000000 P a18a2b9934b94aea9706586a7de4c9cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a18a2b9934b94aea9706586a7de4c9cd; no voters:
I20260808 02:22:41.865828 23267 leader_election.cc:288] T 00000000000000000000000000000000 P a18a2b9934b94aea9706586a7de4c9cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:22:41.865962 23287 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a18a2b9934b94aea9706586a7de4c9cd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:41.866235 23287 raft_consensus.cc:686] T 00000000000000000000000000000000 P a18a2b9934b94aea9706586a7de4c9cd [term 1 LEADER]: Becoming Leader. State: Replica: a18a2b9934b94aea9706586a7de4c9cd, State: Running, Role: LEADER
I20260808 02:22:41.866657 23287 consensus_queue.cc:237] T 00000000000000000000000000000000 P a18a2b9934b94aea9706586a7de4c9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a18a2b9934b94aea9706586a7de4c9cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37697 } }
I20260808 02:22:41.867412 23267 sys_catalog.cc:564] T 00000000000000000000000000000000 P a18a2b9934b94aea9706586a7de4c9cd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:41.867924 23275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:41.859791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:35621 (local address 127.0.0.1:37697)
0808 02:22:41.859923 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:41.859927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:41.863042 (+ 3115us) server_negotiation.cc:407] Connection header received
0808 02:22:41.863196 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:41.863199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:41.863263 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:41.863371 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:41.863816 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:41.864555 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:41.865599 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:41.865978 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:41.866091 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:41.867386 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:41.867390 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:41.867393 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:41.867411 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:41.867448 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:41.867463 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:41.867554 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:41.867731 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 02:22:41.867784 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 02:22:41.868733 23289 sys_catalog.cc:455] T 00000000000000000000000000000000 P a18a2b9934b94aea9706586a7de4c9cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a18a2b9934b94aea9706586a7de4c9cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a18a2b9934b94aea9706586a7de4c9cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37697 } } }
I20260808 02:22:41.868862 23289 sys_catalog.cc:458] T 00000000000000000000000000000000 P a18a2b9934b94aea9706586a7de4c9cd [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:41.869153 23288 sys_catalog.cc:455] T 00000000000000000000000000000000 P a18a2b9934b94aea9706586a7de4c9cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader a18a2b9934b94aea9706586a7de4c9cd. Latest consensus state: current_term: 1 leader_uuid: "a18a2b9934b94aea9706586a7de4c9cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a18a2b9934b94aea9706586a7de4c9cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37697 } } }
I20260808 02:22:41.869251 23288 sys_catalog.cc:458] T 00000000000000000000000000000000 P a18a2b9934b94aea9706586a7de4c9cd [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:41.869541 23298 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:22:41.873118 23298 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:22:41.880481 23298 catalog_manager.cc:1347] Generated new cluster ID: 8c555f1cabc44a8f83fab36bec7da765
I20260808 02:22:41.880642 23298 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:22:41.901434 23298 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:22:41.902552 23298 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:22:41.918336 23298 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a18a2b9934b94aea9706586a7de4c9cd: Generated new TSK 0
I20260808 02:22:41.919353 23298 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:22:41.931208 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 23163
W20260808 02:22:41.949311 23142 connection.cc:535] client connection to 127.0.0.1:37697 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37697): Connection reset by peer (error 104)
2026-08-08T02:22:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1615 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:22:42 dist-test-slave-1vmg krb5kdc[23314](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:22:42 dist-test-slave-1vmg krb5kdc[23314](info): set up 2 sockets
Aug 08 02:22:42 dist-test-slave-1vmg krb5kdc[23314](info): commencing operation
krb5kdc: starting...
W20260808 02:22:43.092844 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.006s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:22:43 dist-test-slave-1vmg krb5kdc[23314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:22:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:22:43.243510 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37127
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:43.369087 23330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:43.369433 23330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:43.369496 23330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:43.373842 23330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:43.373955 23330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:43.373983 23330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:43.374006 23330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:43.374024 23330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:43.380033 23330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:43.381309 23330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:43.382431 23330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:43.392282 23337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:43.392299 23340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:43.392344 23338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:43.392751 23330 server_base.cc:1034] running on GCE node
I20260808 02:22:43.393425 23330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:43.394727 23330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:43.395955 23330 hybrid_clock.cc:648] HybridClock initialized: now 1786155763395890 us; error 76 us; skew 500 ppm
Aug 08 02:22:43 dist-test-slave-1vmg krb5kdc[23314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155763, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:43.399629 23330 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:22:43.400872 23330 webserver.cc:460] Webserver started at http://127.0.0.1:36207/ using document root <none> and password file <none>
I20260808 02:22:43.401557 23330 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:43.401659 23330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:43.401907 23330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:43.404435 23330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "3ffb475aab3b4b2db421131f7c3817fb"
format_stamp: "Formatted at 2026-08-08 02:22:43 on dist-test-slave-1vmg"
server_key: "f3dc9a69b8791c03ae64d77904ff537d"
server_key_iv: "76fde2827e7c834ec2c9a136dfe57ebb"
server_key_version: "encryptionkey@0"
I20260808 02:22:43.405130 23330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "3ffb475aab3b4b2db421131f7c3817fb"
format_stamp: "Formatted at 2026-08-08 02:22:43 on dist-test-slave-1vmg"
server_key: "f3dc9a69b8791c03ae64d77904ff537d"
server_key_iv: "76fde2827e7c834ec2c9a136dfe57ebb"
server_key_version: "encryptionkey@0"
I20260808 02:22:43.409825 23330 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:22:43.413012 23351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:43.414625 23330 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:22:43.414791 23330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "3ffb475aab3b4b2db421131f7c3817fb"
format_stamp: "Formatted at 2026-08-08 02:22:43 on dist-test-slave-1vmg"
server_key: "f3dc9a69b8791c03ae64d77904ff537d"
server_key_iv: "76fde2827e7c834ec2c9a136dfe57ebb"
server_key_version: "encryptionkey@0"
I20260808 02:22:43.414968 23330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:43.420457 23330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:43.424292 23330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:43.424527 23330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:43.433274 23432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37127 every 8 connection(s)
I20260808 02:22:43.433323 23330 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37127
I20260808 02:22:43.434729 23330 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:43.438220 23433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:43.442648 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 23330
I20260808 02:22:43.442811 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:43.443161 24895 external_mini_cluster.cc:1428] Setting key d9f6b04392533629844efd532ed57957
I20260808 02:22:43.445617 23433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ffb475aab3b4b2db421131f7c3817fb: Bootstrap starting.
I20260808 02:22:43.448920 23433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ffb475aab3b4b2db421131f7c3817fb: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:43.449970 23433 log.cc:824] T 00000000000000000000000000000000 P 3ffb475aab3b4b2db421131f7c3817fb: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:43.450048 23437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:43.445059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:56721 (local address 127.0.0.1:37127)
0808 02:22:43.445582 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:43.445603 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:43.445654 (+ 51us) server_negotiation.cc:407] Connection header received
0808 02:22:43.446574 (+ 920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:43.446602 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:43.447051 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:43.447529 (+ 478us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:43.448388 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:43.448394 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:43.448411 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:43.448425 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:43.448497 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:43.448525 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:43.448640 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:43.448783 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 02:22:43.448978 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":142,"threads_started":1}
I20260808 02:22:43.452507 23433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ffb475aab3b4b2db421131f7c3817fb: No bootstrap required, opened a new log
I20260808 02:22:43.453649 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:22:43.457028 23433 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ffb475aab3b4b2db421131f7c3817fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ffb475aab3b4b2db421131f7c3817fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37127 } }
I20260808 02:22:43.457376 23433 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ffb475aab3b4b2db421131f7c3817fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:43.457448 23433 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ffb475aab3b4b2db421131f7c3817fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ffb475aab3b4b2db421131f7c3817fb, State: Initialized, Role: FOLLOWER
I20260808 02:22:43.457800 23437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:43.455990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:41819 (local address 127.0.0.1:37127)
0808 02:22:43.456120 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:43.456125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:43.456327 (+ 202us) server_negotiation.cc:407] Connection header received
0808 02:22:43.456460 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:43.456464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:43.456501 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:43.456570 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:43.457362 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:43.457365 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:43.457367 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:43.457378 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:43.457411 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:43.457424 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:43.457513 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:43.457630 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 02:22:43.457673 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 02:22:43.458029 23433 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ffb475aab3b4b2db421131f7c3817fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ffb475aab3b4b2db421131f7c3817fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37127 } }
I20260808 02:22:43.458195 23433 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ffb475aab3b4b2db421131f7c3817fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:43.458271 23433 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ffb475aab3b4b2db421131f7c3817fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:43.458381 23433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ffb475aab3b4b2db421131f7c3817fb [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:43.459494 23433 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ffb475aab3b4b2db421131f7c3817fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ffb475aab3b4b2db421131f7c3817fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37127 } }
I20260808 02:22:43.459892 23433 leader_election.cc:302] T 00000000000000000000000000000000 P 3ffb475aab3b4b2db421131f7c3817fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ffb475aab3b4b2db421131f7c3817fb; no voters:
I20260808 02:22:43.460246 23433 leader_election.cc:288] T 00000000000000000000000000000000 P 3ffb475aab3b4b2db421131f7c3817fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:22:43.460378 23449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ffb475aab3b4b2db421131f7c3817fb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:43.460603 23449 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ffb475aab3b4b2db421131f7c3817fb [term 1 LEADER]: Becoming Leader. State: Replica: 3ffb475aab3b4b2db421131f7c3817fb, State: Running, Role: LEADER
I20260808 02:22:43.460963 23449 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ffb475aab3b4b2db421131f7c3817fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ffb475aab3b4b2db421131f7c3817fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37127 } }
I20260808 02:22:43.461627 23433 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ffb475aab3b4b2db421131f7c3817fb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:43.462860 23450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ffb475aab3b4b2db421131f7c3817fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ffb475aab3b4b2db421131f7c3817fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ffb475aab3b4b2db421131f7c3817fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37127 } } }
I20260808 02:22:43.463045 23450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ffb475aab3b4b2db421131f7c3817fb [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:43.463335 23451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ffb475aab3b4b2db421131f7c3817fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ffb475aab3b4b2db421131f7c3817fb. Latest consensus state: current_term: 1 leader_uuid: "3ffb475aab3b4b2db421131f7c3817fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ffb475aab3b4b2db421131f7c3817fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37127 } } }
I20260808 02:22:43.463426 23451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ffb475aab3b4b2db421131f7c3817fb [sys.catalog]: This master's current role is: LEADER
W20260808 02:22:43.467463 23473 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3ffb475aab3b4b2db421131f7c3817fb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:22:43.467571 23473 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:22:43.467672 23461 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:22:43.468777 23461 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:22:43.475379 23461 catalog_manager.cc:1347] Generated new cluster ID: 7129586ed1414e14954c960517af062d
I20260808 02:22:43.475485 23461 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:22:43.501016 23461 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:22:43.501480 23461 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:22:43.509610 23461 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ffb475aab3b4b2db421131f7c3817fb: Generated new TSK 0
I20260808 02:22:43.510627 23461 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:22:43.517328 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 23330
2026-08-08T02:22:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1579 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:22:43 dist-test-slave-1vmg krb5kdc[23483](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:22:43 dist-test-slave-1vmg krb5kdc[23483](info): set up 2 sockets
Aug 08 02:22:43 dist-test-slave-1vmg krb5kdc[23483](info): commencing operation
krb5kdc: starting...
W20260808 02:22:44.668046 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:22:44 dist-test-slave-1vmg krb5kdc[23483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:22:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:22:44.814158 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42483
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42483
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:44.940094 23501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:44.940420 23501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:44.940464 23501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:44.944746 23501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:44.944830 23501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:44.944849 23501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:44.944865 23501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:44.944880 23501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:44.950632 23501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42483
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:44.951956 23501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:44.953122 23501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:44.962594 23513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:44.962598 23509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:44.962594 23511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:44.962646 23501 server_base.cc:1034] running on GCE node
I20260808 02:22:44.963541 23501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:44.964850 23501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:44.966073 23501 hybrid_clock.cc:648] HybridClock initialized: now 1786155764966051 us; error 38 us; skew 500 ppm
Aug 08 02:22:44 dist-test-slave-1vmg krb5kdc[23483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155764, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:44.972779 23501 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:22:44.974061 23501 webserver.cc:460] Webserver started at http://127.0.0.1:33809/ using document root <none> and password file <none>
I20260808 02:22:44.974836 23501 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:44.974962 23501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:44.975234 23501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:44.977713 23501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "84933cf3d2d142a19dbd30c04f22a706"
format_stamp: "Formatted at 2026-08-08 02:22:44 on dist-test-slave-1vmg"
server_key: "358a4444f11c2eb594eac25069b0daa2"
server_key_iv: "ce87aa9609643ff73c6c6487b7ad5707"
server_key_version: "encryptionkey@0"
I20260808 02:22:44.978418 23501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "84933cf3d2d142a19dbd30c04f22a706"
format_stamp: "Formatted at 2026-08-08 02:22:44 on dist-test-slave-1vmg"
server_key: "358a4444f11c2eb594eac25069b0daa2"
server_key_iv: "ce87aa9609643ff73c6c6487b7ad5707"
server_key_version: "encryptionkey@0"
I20260808 02:22:44.983067 23501 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 02:22:44.986189 23522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:44.987908 23501 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:22:44.988103 23501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "84933cf3d2d142a19dbd30c04f22a706"
format_stamp: "Formatted at 2026-08-08 02:22:44 on dist-test-slave-1vmg"
server_key: "358a4444f11c2eb594eac25069b0daa2"
server_key_iv: "ce87aa9609643ff73c6c6487b7ad5707"
server_key_version: "encryptionkey@0"
I20260808 02:22:44.988284 23501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:45.012486 23501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:45.016165 23501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:45.016407 23501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:45.024809 23501 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42483
I20260808 02:22:45.024837 23599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42483 every 8 connection(s)
I20260808 02:22:45.025928 23501 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:45.029414 23600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:45.033985 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 23501
I20260808 02:22:45.034143 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:45.034440 24895 external_mini_cluster.cc:1428] Setting key 1fa06e6edb36049fbec0e87a439af088
I20260808 02:22:45.037271 23600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84933cf3d2d142a19dbd30c04f22a706: Bootstrap starting.
I20260808 02:22:45.039981 23600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84933cf3d2d142a19dbd30c04f22a706: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:45.041035 23600 log.cc:824] T 00000000000000000000000000000000 P 84933cf3d2d142a19dbd30c04f22a706: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:45.043507 23600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84933cf3d2d142a19dbd30c04f22a706: No bootstrap required, opened a new log
Aug 08 02:22:45 dist-test-slave-1vmg krb5kdc[23483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 02:22:45.047745 23600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84933cf3d2d142a19dbd30c04f22a706 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84933cf3d2d142a19dbd30c04f22a706" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42483 } }
I20260808 02:22:45.048029 23600 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84933cf3d2d142a19dbd30c04f22a706 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:45.048094 23600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84933cf3d2d142a19dbd30c04f22a706 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84933cf3d2d142a19dbd30c04f22a706, State: Initialized, Role: FOLLOWER
I20260808 02:22:45.048612 23600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84933cf3d2d142a19dbd30c04f22a706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84933cf3d2d142a19dbd30c04f22a706" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42483 } }
I20260808 02:22:45.048789 23600 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84933cf3d2d142a19dbd30c04f22a706 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:45.048862 23600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84933cf3d2d142a19dbd30c04f22a706 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:45.048971 23600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84933cf3d2d142a19dbd30c04f22a706 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:45.050071 23600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84933cf3d2d142a19dbd30c04f22a706 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84933cf3d2d142a19dbd30c04f22a706" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42483 } }
I20260808 02:22:45.050459 23600 leader_election.cc:302] T 00000000000000000000000000000000 P 84933cf3d2d142a19dbd30c04f22a706 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84933cf3d2d142a19dbd30c04f22a706; no voters:
I20260808 02:22:45.050802 23600 leader_election.cc:288] T 00000000000000000000000000000000 P 84933cf3d2d142a19dbd30c04f22a706 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:22:45.050973 23607 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84933cf3d2d142a19dbd30c04f22a706 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:45.051028 23604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:45.036114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:52761 (local address 127.0.0.1:42483)
0808 02:22:45.036729 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:45.036748 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:45.036795 (+ 47us) server_negotiation.cc:407] Connection header received
0808 02:22:45.037551 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:45.037573 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:45.037986 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:45.038329 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:45.039293 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:45.040632 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:45.041622 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:45.042165 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:45.042325 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:45.045437 (+ 3112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:45.045443 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:45.045461 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:45.045496 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:45.047463 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:45.048057 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:45.048065 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:45.048072 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:45.048243 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:45.048593 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:45.048597 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:45.048599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:45.049152 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:45.049377 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:45.049763 (+ 386us) server_negotiation.cc:299] Negotiation successful
0808 02:22:45.050015 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":160,"threads_started":1}
I20260808 02:22:45.051236 23607 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84933cf3d2d142a19dbd30c04f22a706 [term 1 LEADER]: Becoming Leader. State: Replica: 84933cf3d2d142a19dbd30c04f22a706, State: Running, Role: LEADER
I20260808 02:22:45.051579 23607 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84933cf3d2d142a19dbd30c04f22a706 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84933cf3d2d142a19dbd30c04f22a706" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42483 } }
I20260808 02:22:45.052320 23600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84933cf3d2d142a19dbd30c04f22a706 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:45.052933 23607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84933cf3d2d142a19dbd30c04f22a706 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84933cf3d2d142a19dbd30c04f22a706. Latest consensus state: current_term: 1 leader_uuid: "84933cf3d2d142a19dbd30c04f22a706" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84933cf3d2d142a19dbd30c04f22a706" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42483 } } }
I20260808 02:22:45.053053 23607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84933cf3d2d142a19dbd30c04f22a706 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:45.053361 23608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84933cf3d2d142a19dbd30c04f22a706 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84933cf3d2d142a19dbd30c04f22a706" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84933cf3d2d142a19dbd30c04f22a706" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42483 } } }
I20260808 02:22:45.053467 23608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84933cf3d2d142a19dbd30c04f22a706 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:45.054023 23617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:22:45.054677 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:22:45.058032 23617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:22:45.065078 23617 catalog_manager.cc:1347] Generated new cluster ID: cf201ae6c8084bb9b17d227421782279
I20260808 02:22:45.065267 23617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:22:45.071728 23604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:45.057010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:58611 (local address 127.0.0.1:42483)
0808 02:22:45.057133 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:45.057139 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:45.063402 (+ 6263us) server_negotiation.cc:407] Connection header received
0808 02:22:45.063565 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:45.063569 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:45.063632 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:45.063738 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:45.064748 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:45.065498 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:45.066393 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:45.066663 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:45.066755 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:45.068310 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:45.068314 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:45.068316 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:45.068344 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:45.069759 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:45.070300 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:45.070303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:45.070305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:45.070391 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:45.070722 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:45.070726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:45.070727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:45.071001 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:45.071171 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:45.071442 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 02:22:45.071549 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 02:22:45.088171 23617 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:22:45.089233 23617 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:22:45.096658 23617 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84933cf3d2d142a19dbd30c04f22a706: Generated new TSK 0
I20260808 02:22:45.097484 23617 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:22:45.112867 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 23501
W20260808 02:22:45.130563 23479 connection.cc:535] client connection to 127.0.0.1:42483 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42483): Connection reset by peer (error 104)
2026-08-08T02:22:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1601 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:22:45 dist-test-slave-1vmg krb5kdc[23648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:22:45 dist-test-slave-1vmg krb5kdc[23648](info): set up 2 sockets
Aug 08 02:22:45 dist-test-slave-1vmg krb5kdc[23648](info): commencing operation
krb5kdc: starting...
W20260808 02:22:46.272893 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:22:46 dist-test-slave-1vmg krb5kdc[23648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:22:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:22:46.422959 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44079
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:46.553601 23669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:46.553978 23669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:46.554044 23669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:46.558347 23669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:46.558451 23669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:46.558482 23669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:46.558506 23669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:46.558526 23669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:46.564541 23669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44079
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:46.565842 23669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:46.566987 23669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:46.576318 23678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:46.576377 23683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:46.576346 23679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:46.576668 23669 server_base.cc:1034] running on GCE node
I20260808 02:22:46.577353 23669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:46.578629 23669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:46.579833 23669 hybrid_clock.cc:648] HybridClock initialized: now 1786155766579799 us; error 47 us; skew 500 ppm
Aug 08 02:22:46 dist-test-slave-1vmg krb5kdc[23648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155766, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:46.583207 23669 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:22:46.584348 23669 webserver.cc:460] Webserver started at http://127.0.0.1:42969/ using document root <none> and password file <none>
I20260808 02:22:46.584995 23669 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:46.585078 23669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:46.585361 23669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:46.588194 23669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "79d7a985d8554c67903b2bcd639092d8"
format_stamp: "Formatted at 2026-08-08 02:22:46 on dist-test-slave-1vmg"
server_key: "31ac346960973532d7678bd974079095"
server_key_iv: "d3e3a4ce359e40d0e3c32d18c1f0562c"
server_key_version: "encryptionkey@0"
I20260808 02:22:46.589084 23669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "79d7a985d8554c67903b2bcd639092d8"
format_stamp: "Formatted at 2026-08-08 02:22:46 on dist-test-slave-1vmg"
server_key: "31ac346960973532d7678bd974079095"
server_key_iv: "d3e3a4ce359e40d0e3c32d18c1f0562c"
server_key_version: "encryptionkey@0"
I20260808 02:22:46.593804 23669 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:22:46.596822 23694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:46.598428 23669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:22:46.598616 23669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "79d7a985d8554c67903b2bcd639092d8"
format_stamp: "Formatted at 2026-08-08 02:22:46 on dist-test-slave-1vmg"
server_key: "31ac346960973532d7678bd974079095"
server_key_iv: "d3e3a4ce359e40d0e3c32d18c1f0562c"
server_key_version: "encryptionkey@0"
I20260808 02:22:46.598745 23669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:46.626389 23669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:46.630211 23669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:46.630431 23669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:46.638469 23669 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44079
I20260808 02:22:46.638496 23778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44079 every 8 connection(s)
I20260808 02:22:46.639719 23669 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:46.642971 23780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:46.643246 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 23669
I20260808 02:22:46.643399 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:46.643692 24895 external_mini_cluster.cc:1428] Setting key 1b861e434abd1f18fd4da1f35e2dbabf
I20260808 02:22:46.651284 23780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79d7a985d8554c67903b2bcd639092d8: Bootstrap starting.
I20260808 02:22:46.654273 23780 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79d7a985d8554c67903b2bcd639092d8: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:46.655207 23786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:46.645206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:37425 (local address 127.0.0.1:44079)
0808 02:22:46.645712 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:46.645735 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:46.645785 (+ 50us) server_negotiation.cc:407] Connection header received
0808 02:22:46.646753 (+ 968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:46.646782 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:46.647329 (+ 547us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:46.647766 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:46.648259 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:46.649786 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:46.650861 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:46.651637 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:46.651815 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:46.652874 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:46.652881 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:46.652900 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:46.653442 (+ 542us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:46.653527 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:46.653563 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:46.653721 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:46.653960 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 02:22:46.654122 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":133,"threads_started":1}
I20260808 02:22:46.655300 23780 log.cc:824] T 00000000000000000000000000000000 P 79d7a985d8554c67903b2bcd639092d8: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:46.657665 23780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79d7a985d8554c67903b2bcd639092d8: No bootstrap required, opened a new log
I20260808 02:22:46.658488 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:22:46.662014 23780 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79d7a985d8554c67903b2bcd639092d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d7a985d8554c67903b2bcd639092d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44079 } }
I20260808 02:22:46.662384 23780 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79d7a985d8554c67903b2bcd639092d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:46.662447 23780 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79d7a985d8554c67903b2bcd639092d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79d7a985d8554c67903b2bcd639092d8, State: Initialized, Role: FOLLOWER
I20260808 02:22:46.662976 23780 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79d7a985d8554c67903b2bcd639092d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d7a985d8554c67903b2bcd639092d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44079 } }
I20260808 02:22:46.663190 23780 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79d7a985d8554c67903b2bcd639092d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:46.663270 23780 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79d7a985d8554c67903b2bcd639092d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:46.663388 23780 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79d7a985d8554c67903b2bcd639092d8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:46.664690 23780 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79d7a985d8554c67903b2bcd639092d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d7a985d8554c67903b2bcd639092d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44079 } }
I20260808 02:22:46.665164 23780 leader_election.cc:302] T 00000000000000000000000000000000 P 79d7a985d8554c67903b2bcd639092d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79d7a985d8554c67903b2bcd639092d8; no voters:
I20260808 02:22:46.665644 23780 leader_election.cc:288] T 00000000000000000000000000000000 P 79d7a985d8554c67903b2bcd639092d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:22:46.665704 23786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:46.660782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:45591 (local address 127.0.0.1:44079)
0808 02:22:46.660914 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:46.660919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:46.661047 (+ 128us) server_negotiation.cc:407] Connection header received
0808 02:22:46.661159 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:46.661162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:46.661217 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:46.661313 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:46.661646 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:46.662379 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:46.663487 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:46.663837 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:46.663950 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:46.665043 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:46.665047 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:46.665050 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:46.665066 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:46.665101 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:46.665115 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:46.665221 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:46.665505 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 02:22:46.665551 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:22:46.665776 23795 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79d7a985d8554c67903b2bcd639092d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:46.666046 23795 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79d7a985d8554c67903b2bcd639092d8 [term 1 LEADER]: Becoming Leader. State: Replica: 79d7a985d8554c67903b2bcd639092d8, State: Running, Role: LEADER
I20260808 02:22:46.666433 23795 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79d7a985d8554c67903b2bcd639092d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d7a985d8554c67903b2bcd639092d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44079 } }
I20260808 02:22:46.667140 23780 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79d7a985d8554c67903b2bcd639092d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:46.667914 23795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d7a985d8554c67903b2bcd639092d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79d7a985d8554c67903b2bcd639092d8. Latest consensus state: current_term: 1 leader_uuid: "79d7a985d8554c67903b2bcd639092d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d7a985d8554c67903b2bcd639092d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44079 } } }
I20260808 02:22:46.668057 23795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d7a985d8554c67903b2bcd639092d8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:46.668383 23797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d7a985d8554c67903b2bcd639092d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79d7a985d8554c67903b2bcd639092d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d7a985d8554c67903b2bcd639092d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44079 } } }
I20260808 02:22:46.668496 23797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d7a985d8554c67903b2bcd639092d8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:46.668479 23806 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:22:46.672574 23806 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:22:46.679261 23806 catalog_manager.cc:1347] Generated new cluster ID: 6bd8addb2c234caaad270551f2b622bd
I20260808 02:22:46.679371 23806 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:22:46.693915 23806 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:22:46.695147 23806 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:22:46.702553 23806 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79d7a985d8554c67903b2bcd639092d8: Generated new TSK 0
I20260808 02:22:46.703655 23806 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:22:46.710866 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 23669
W20260808 02:22:46.726455 23646 connection.cc:535] client connection to 127.0.0.1:44079 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44079): Connection reset by peer (error 104)
2026-08-08T02:22:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1595 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:22:46 dist-test-slave-1vmg krb5kdc[23822](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:22:46 dist-test-slave-1vmg krb5kdc[23822](info): set up 2 sockets
Aug 08 02:22:46 dist-test-slave-1vmg krb5kdc[23822](info): commencing operation
krb5kdc: starting...
W20260808 02:22:47.870018 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:22:47 dist-test-slave-1vmg krb5kdc[23822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:22:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:22:48.021045 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38369
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:48.145874 23843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:48.146241 23843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:48.146310 23843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:48.151024 23843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:48.151134 23843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:48.151165 23843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:48.151197 23843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:48.151222 23843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:48.157581 23843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38369
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:48.158957 23843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:48.160231 23843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:48.170454 23861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:48.170459 23854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:48.170454 23853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:48.170892 23843 server_base.cc:1034] running on GCE node
I20260808 02:22:48.171727 23843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:48.173116 23843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:48.174340 23843 hybrid_clock.cc:648] HybridClock initialized: now 1786155768174317 us; error 41 us; skew 500 ppm
Aug 08 02:22:48 dist-test-slave-1vmg krb5kdc[23822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155768, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:48.178059 23843 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:22:48.179332 23843 webserver.cc:460] Webserver started at http://127.0.0.1:34641/ using document root <none> and password file <none>
I20260808 02:22:48.180023 23843 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:48.180097 23843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:48.180320 23843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:48.183089 23843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "c249d4432aad4745abfc86ed2a84527b"
format_stamp: "Formatted at 2026-08-08 02:22:48 on dist-test-slave-1vmg"
server_key: "28fc374cc9c61af9002aed34c4c39add"
server_key_iv: "f6a2a146200c45a2c6dcdb7d65dc379a"
server_key_version: "encryptionkey@0"
I20260808 02:22:48.183777 23843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "c249d4432aad4745abfc86ed2a84527b"
format_stamp: "Formatted at 2026-08-08 02:22:48 on dist-test-slave-1vmg"
server_key: "28fc374cc9c61af9002aed34c4c39add"
server_key_iv: "f6a2a146200c45a2c6dcdb7d65dc379a"
server_key_version: "encryptionkey@0"
I20260808 02:22:48.188382 23843 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:22:48.191655 23869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:48.193329 23843 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:22:48.193506 23843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "c249d4432aad4745abfc86ed2a84527b"
format_stamp: "Formatted at 2026-08-08 02:22:48 on dist-test-slave-1vmg"
server_key: "28fc374cc9c61af9002aed34c4c39add"
server_key_iv: "f6a2a146200c45a2c6dcdb7d65dc379a"
server_key_version: "encryptionkey@0"
I20260808 02:22:48.193673 23843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:48.199564 23843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:48.203770 23843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:48.203960 23843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:48.211643 23843 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38369
I20260808 02:22:48.211663 23947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38369 every 8 connection(s)
I20260808 02:22:48.212721 23843 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:48.215859 23948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:48.220340 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 23843
I20260808 02:22:48.220477 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:48.220754 24895 external_mini_cluster.cc:1428] Setting key 02d61d66e3ec30d32a00c71eeee9b0f7
I20260808 02:22:48.222385 23948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c249d4432aad4745abfc86ed2a84527b: Bootstrap starting.
I20260808 02:22:48.225028 23948 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c249d4432aad4745abfc86ed2a84527b: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:48.226143 23948 log.cc:824] T 00000000000000000000000000000000 P c249d4432aad4745abfc86ed2a84527b: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:48.226879 23951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:48.222411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:60247 (local address 127.0.0.1:38369)
0808 02:22:48.222990 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:48.223007 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:48.223052 (+ 45us) server_negotiation.cc:407] Connection header received
0808 02:22:48.223835 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:48.223856 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:48.224191 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:48.224552 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:48.225536 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:48.225541 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:48.225556 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:48.225569 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:48.225634 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:48.225656 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:48.225759 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:48.225909 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 02:22:48.226092 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":153,"threads_started":1}
I20260808 02:22:48.228528 23948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c249d4432aad4745abfc86ed2a84527b: No bootstrap required, opened a new log
I20260808 02:22:48.230069 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:22:48.233475 23948 raft_consensus.cc:348] T 00000000000000000000000000000000 P c249d4432aad4745abfc86ed2a84527b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c249d4432aad4745abfc86ed2a84527b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38369 } }
I20260808 02:22:48.233767 23948 raft_consensus.cc:374] T 00000000000000000000000000000000 P c249d4432aad4745abfc86ed2a84527b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:48.233831 23948 raft_consensus.cc:729] T 00000000000000000000000000000000 P c249d4432aad4745abfc86ed2a84527b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c249d4432aad4745abfc86ed2a84527b, State: Initialized, Role: FOLLOWER
I20260808 02:22:48.234382 23948 consensus_queue.cc:260] T 00000000000000000000000000000000 P c249d4432aad4745abfc86ed2a84527b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c249d4432aad4745abfc86ed2a84527b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38369 } }
I20260808 02:22:48.234535 23951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:48.232712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:58199 (local address 127.0.0.1:38369)
0808 02:22:48.232869 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:48.232874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:48.232964 (+ 90us) server_negotiation.cc:407] Connection header received
0808 02:22:48.233066 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:48.233068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:48.233110 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:48.233232 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:48.234085 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:48.234089 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:48.234092 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:48.234105 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:48.234140 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:48.234153 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:48.234247 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:48.234332 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 02:22:48.234377 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 02:22:48.234588 23948 raft_consensus.cc:388] T 00000000000000000000000000000000 P c249d4432aad4745abfc86ed2a84527b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:48.234654 23948 raft_consensus.cc:482] T 00000000000000000000000000000000 P c249d4432aad4745abfc86ed2a84527b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:48.234776 23948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c249d4432aad4745abfc86ed2a84527b [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:48.236007 23948 raft_consensus.cc:504] T 00000000000000000000000000000000 P c249d4432aad4745abfc86ed2a84527b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c249d4432aad4745abfc86ed2a84527b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38369 } }
I20260808 02:22:48.236380 23948 leader_election.cc:302] T 00000000000000000000000000000000 P c249d4432aad4745abfc86ed2a84527b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c249d4432aad4745abfc86ed2a84527b; no voters:
I20260808 02:22:48.236708 23948 leader_election.cc:288] T 00000000000000000000000000000000 P c249d4432aad4745abfc86ed2a84527b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:22:48.236822 23959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c249d4432aad4745abfc86ed2a84527b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:48.237075 23959 raft_consensus.cc:686] T 00000000000000000000000000000000 P c249d4432aad4745abfc86ed2a84527b [term 1 LEADER]: Becoming Leader. State: Replica: c249d4432aad4745abfc86ed2a84527b, State: Running, Role: LEADER
I20260808 02:22:48.237406 23959 consensus_queue.cc:237] T 00000000000000000000000000000000 P c249d4432aad4745abfc86ed2a84527b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c249d4432aad4745abfc86ed2a84527b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38369 } }
I20260808 02:22:48.238036 23948 sys_catalog.cc:564] T 00000000000000000000000000000000 P c249d4432aad4745abfc86ed2a84527b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:48.239166 23961 sys_catalog.cc:455] T 00000000000000000000000000000000 P c249d4432aad4745abfc86ed2a84527b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c249d4432aad4745abfc86ed2a84527b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c249d4432aad4745abfc86ed2a84527b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38369 } } }
I20260808 02:22:48.239315 23961 sys_catalog.cc:458] T 00000000000000000000000000000000 P c249d4432aad4745abfc86ed2a84527b [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:48.239609 23963 sys_catalog.cc:455] T 00000000000000000000000000000000 P c249d4432aad4745abfc86ed2a84527b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c249d4432aad4745abfc86ed2a84527b. Latest consensus state: current_term: 1 leader_uuid: "c249d4432aad4745abfc86ed2a84527b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c249d4432aad4745abfc86ed2a84527b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38369 } } }
I20260808 02:22:48.239722 23963 sys_catalog.cc:458] T 00000000000000000000000000000000 P c249d4432aad4745abfc86ed2a84527b [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:48.239864 23973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:22:48.243783 23973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:22:48.250379 23973 catalog_manager.cc:1347] Generated new cluster ID: 35bc52d178b44b3aaf75bd7fa68bea7e
I20260808 02:22:48.250491 23973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:22:48.258512 23973 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:22:48.259053 23973 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:22:48.266109 23973 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c249d4432aad4745abfc86ed2a84527b: Generated new TSK 0
I20260808 02:22:48.267190 23973 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:22:48.274135 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 23843
2026-08-08T02:22:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1557 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:22:48 dist-test-slave-1vmg krb5kdc[23992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:22:48 dist-test-slave-1vmg krb5kdc[23992](info): set up 2 sockets
Aug 08 02:22:48 dist-test-slave-1vmg krb5kdc[23992](info): commencing operation
krb5kdc: starting...
W20260808 02:22:49.424609 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:22:49 dist-test-slave-1vmg krb5kdc[23992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:22:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:22:49.579572 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:38615
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:39171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:38615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:49.705927 24013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:49.706272 24013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:49.706317 24013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:49.710717 24013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:49.710842 24013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:49.710863 24013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:49.710884 24013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:49.710899 24013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:49.716897 24013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:39171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:38615
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:38615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:49.718217 24013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:49.719421 24013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:49.729498 24027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:49.729485 24022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:49.729485 24020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:49.730017 24013 server_base.cc:1034] running on GCE node
I20260808 02:22:49.730691 24013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:49.732036 24013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:49.733228 24013 hybrid_clock.cc:648] HybridClock initialized: now 1786155769733203 us; error 42 us; skew 500 ppm
Aug 08 02:22:49 dist-test-slave-1vmg krb5kdc[23992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155769, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:49.736336 24013 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:22:49.737434 24013 webserver.cc:460] Webserver started at http://127.24.79.254:34311/ using document root <none> and password file <none>
I20260808 02:22:49.738085 24013 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:49.738161 24013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:49.738387 24013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:49.740882 24013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "80c708bb1c9d4a7cbdc710a6076d61e1"
format_stamp: "Formatted at 2026-08-08 02:22:49 on dist-test-slave-1vmg"
server_key: "5f485932ba7bf05b72a3f5d25ec119ec"
server_key_iv: "89e41008a565a92596e68952e9ebcf87"
server_key_version: "encryptionkey@0"
I20260808 02:22:49.741577 24013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "80c708bb1c9d4a7cbdc710a6076d61e1"
format_stamp: "Formatted at 2026-08-08 02:22:49 on dist-test-slave-1vmg"
server_key: "5f485932ba7bf05b72a3f5d25ec119ec"
server_key_iv: "89e41008a565a92596e68952e9ebcf87"
server_key_version: "encryptionkey@0"
I20260808 02:22:49.746219 24013 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:22:49.749369 24039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:49.750870 24013 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:22:49.751078 24013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "80c708bb1c9d4a7cbdc710a6076d61e1"
format_stamp: "Formatted at 2026-08-08 02:22:49 on dist-test-slave-1vmg"
server_key: "5f485932ba7bf05b72a3f5d25ec119ec"
server_key_iv: "89e41008a565a92596e68952e9ebcf87"
server_key_version: "encryptionkey@0"
I20260808 02:22:49.751219 24013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:49.765254 24013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:49.769384 24013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:49.769608 24013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:49.777699 24013 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:38615
I20260808 02:22:49.777720 24116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:38615 every 8 connection(s)
I20260808 02:22:49.778875 24013 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:49.782341 24119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:49.788895 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 24013
I20260808 02:22:49.789064 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:49.789355 24895 external_mini_cluster.cc:1428] Setting key 756273189051da715889dff874eb33c6
I20260808 02:22:49.789346 24119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80c708bb1c9d4a7cbdc710a6076d61e1: Bootstrap starting.
I20260808 02:22:49.792268 24119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80c708bb1c9d4a7cbdc710a6076d61e1: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:49.793213 24119 log.cc:824] T 00000000000000000000000000000000 P 80c708bb1c9d4a7cbdc710a6076d61e1: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:49.796020 24119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80c708bb1c9d4a7cbdc710a6076d61e1: No bootstrap required, opened a new log
Aug 08 02:22:49 dist-test-slave-1vmg krb5kdc[23992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:22:49.800954 24119 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80c708bb1c9d4a7cbdc710a6076d61e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80c708bb1c9d4a7cbdc710a6076d61e1" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38615 } }
I20260808 02:22:49.801237 24119 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80c708bb1c9d4a7cbdc710a6076d61e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:49.801285 24119 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80c708bb1c9d4a7cbdc710a6076d61e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80c708bb1c9d4a7cbdc710a6076d61e1, State: Initialized, Role: FOLLOWER
I20260808 02:22:49.801847 24119 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80c708bb1c9d4a7cbdc710a6076d61e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80c708bb1c9d4a7cbdc710a6076d61e1" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38615 } }
I20260808 02:22:49.802026 24119 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80c708bb1c9d4a7cbdc710a6076d61e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:49.802105 24119 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80c708bb1c9d4a7cbdc710a6076d61e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:49.802213 24119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80c708bb1c9d4a7cbdc710a6076d61e1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:49.803463 24119 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80c708bb1c9d4a7cbdc710a6076d61e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80c708bb1c9d4a7cbdc710a6076d61e1" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38615 } }
I20260808 02:22:49.803931 24119 leader_election.cc:302] T 00000000000000000000000000000000 P 80c708bb1c9d4a7cbdc710a6076d61e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80c708bb1c9d4a7cbdc710a6076d61e1; no voters:
I20260808 02:22:49.804354 24119 leader_election.cc:288] T 00000000000000000000000000000000 P 80c708bb1c9d4a7cbdc710a6076d61e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:22:49.804479 24129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80c708bb1c9d4a7cbdc710a6076d61e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:49.804706 24129 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80c708bb1c9d4a7cbdc710a6076d61e1 [term 1 LEADER]: Becoming Leader. State: Replica: 80c708bb1c9d4a7cbdc710a6076d61e1, State: Running, Role: LEADER
I20260808 02:22:49.805148 24129 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80c708bb1c9d4a7cbdc710a6076d61e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80c708bb1c9d4a7cbdc710a6076d61e1" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38615 } }
I20260808 02:22:49.805897 24119 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80c708bb1c9d4a7cbdc710a6076d61e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:49.807000 24137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80c708bb1c9d4a7cbdc710a6076d61e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80c708bb1c9d4a7cbdc710a6076d61e1. Latest consensus state: current_term: 1 leader_uuid: "80c708bb1c9d4a7cbdc710a6076d61e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80c708bb1c9d4a7cbdc710a6076d61e1" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38615 } } }
I20260808 02:22:49.807147 24137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80c708bb1c9d4a7cbdc710a6076d61e1 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:49.807169 24125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:49.790825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46113 (local address 127.24.79.254:38615)
0808 02:22:49.791327 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:49.791346 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:49.791387 (+ 41us) server_negotiation.cc:407] Connection header received
0808 02:22:49.792131 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:49.792152 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:49.792573 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:49.792905 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:49.793938 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:49.795178 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:49.796558 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:49.797164 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:49.797323 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:49.800698 (+ 3375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:49.800705 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:49.800726 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:49.800769 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:49.802816 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:49.803523 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:49.803529 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:49.803535 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:49.803705 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:49.804089 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:49.804092 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:49.804093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:49.804694 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:49.804987 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:49.805675 (+ 688us) server_negotiation.cc:299] Negotiation successful
0808 02:22:49.805948 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":139,"threads_started":1}
I20260808 02:22:49.807437 24130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80c708bb1c9d4a7cbdc710a6076d61e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80c708bb1c9d4a7cbdc710a6076d61e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80c708bb1c9d4a7cbdc710a6076d61e1" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 38615 } } }
I20260808 02:22:49.807533 24130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80c708bb1c9d4a7cbdc710a6076d61e1 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:49.810994 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 02:22:49.812255 24154 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 80c708bb1c9d4a7cbdc710a6076d61e1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:22:49.812417 24154 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:22:49.812500 24145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:22:49.813510 24145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:22:49.820838 24145 catalog_manager.cc:1347] Generated new cluster ID: fe72b3f4b2f4414b978582cf22e0aa15
I20260808 02:22:49.820926 24145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:22:49.825227 24125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:49.815618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39555 (local address 127.24.79.254:38615)
0808 02:22:49.815740 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:49.815745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:49.816136 (+ 391us) server_negotiation.cc:407] Connection header received
0808 02:22:49.816359 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:49.816363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:49.816423 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:49.816521 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:49.817510 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:49.818236 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:49.819276 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:49.819544 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:49.819641 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:49.821380 (+ 1739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:49.821384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:49.821387 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:49.821419 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:49.823251 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:49.823866 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:49.823870 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:49.823872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:49.823974 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:49.824287 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:49.824290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:49.824292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:49.824532 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:49.824680 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:49.824924 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 02:22:49.825050 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 02:22:49.829664 24145 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:22:49.831113 24145 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:22:49.844746 24145 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80c708bb1c9d4a7cbdc710a6076d61e1: Generated new TSK 0
I20260808 02:22:49.845822 24145 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:22:49.850528 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 24013
W20260808 02:22:49.872619 23989 connection.cc:535] client connection to 127.24.79.254:38615 recv error: Network error: failed to read from TLS socket (remote: 127.24.79.254:38615): Connection reset by peer (error 104)
2026-08-08T02:22:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1587 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:22:49 dist-test-slave-1vmg krb5kdc[24170](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:22:49 dist-test-slave-1vmg krb5kdc[24170](info): set up 2 sockets
Aug 08 02:22:49 dist-test-slave-1vmg krb5kdc[24170](info): commencing operation
krb5kdc: starting...
W20260808 02:22:51.032167 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:22:51 dist-test-slave-1vmg krb5kdc[24170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:22:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:22:51.184767 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:40125
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:36637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:40125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:51.312745 24186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:51.313068 24186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:51.313122 24186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:51.317358 24186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:51.317461 24186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:51.317479 24186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:51.317498 24186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:51.317512 24186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:51.323758 24186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:36637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:40125
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:40125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:51.325078 24186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:51.326541 24186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:51.338635 24192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:51.338642 24194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:51.338642 24197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:51.338796 24186 server_base.cc:1034] running on GCE node
I20260808 02:22:51.339737 24186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:51.341092 24186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:51.342299 24186 hybrid_clock.cc:648] HybridClock initialized: now 1786155771342277 us; error 41 us; skew 500 ppm
Aug 08 02:22:51 dist-test-slave-1vmg krb5kdc[24170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155771, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:51.345650 24186 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:22:51.346799 24186 webserver.cc:460] Webserver started at http://127.24.79.254:45223/ using document root <none> and password file <none>
I20260808 02:22:51.347486 24186 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:51.347591 24186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:51.347854 24186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:51.350418 24186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "790053ba74214d85bd7938da3c9bbe5f"
format_stamp: "Formatted at 2026-08-08 02:22:51 on dist-test-slave-1vmg"
server_key: "f638b6d3a60978f5c4f1ee714b9e38de"
server_key_iv: "e49e55c9f398da9264e554f0be6e2a5f"
server_key_version: "encryptionkey@0"
I20260808 02:22:51.351217 24186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "790053ba74214d85bd7938da3c9bbe5f"
format_stamp: "Formatted at 2026-08-08 02:22:51 on dist-test-slave-1vmg"
server_key: "f638b6d3a60978f5c4f1ee714b9e38de"
server_key_iv: "e49e55c9f398da9264e554f0be6e2a5f"
server_key_version: "encryptionkey@0"
I20260808 02:22:51.355953 24186 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:22:51.359009 24208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:51.360469 24186 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:22:51.360643 24186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "790053ba74214d85bd7938da3c9bbe5f"
format_stamp: "Formatted at 2026-08-08 02:22:51 on dist-test-slave-1vmg"
server_key: "f638b6d3a60978f5c4f1ee714b9e38de"
server_key_iv: "e49e55c9f398da9264e554f0be6e2a5f"
server_key_version: "encryptionkey@0"
I20260808 02:22:51.360785 24186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:51.405194 24186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:51.409291 24186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:51.409557 24186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:51.418107 24186 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:40125
I20260808 02:22:51.418136 24285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:40125 every 8 connection(s)
I20260808 02:22:51.419369 24186 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:51.423178 24286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:51.425019 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 24186
I20260808 02:22:51.425171 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:51.425480 24895 external_mini_cluster.cc:1428] Setting key dc129cf98c2352dfeedbc45b61b412f4
I20260808 02:22:51.430719 24286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 790053ba74214d85bd7938da3c9bbe5f: Bootstrap starting.
I20260808 02:22:51.433480 24286 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 790053ba74214d85bd7938da3c9bbe5f: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:51.434477 24286 log.cc:824] T 00000000000000000000000000000000 P 790053ba74214d85bd7938da3c9bbe5f: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:51.436645 24291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:51.427368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51881 (local address 127.24.79.254:40125)
0808 02:22:51.427959 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:51.427979 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:51.428026 (+ 47us) server_negotiation.cc:407] Connection header received
0808 02:22:51.428810 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:51.428832 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:51.429255 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:51.429651 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:51.430151 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:51.431331 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:51.432335 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:51.432904 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:51.433095 (+ 191us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:51.434267 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:51.434273 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:51.434291 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:51.434824 (+ 533us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:51.434899 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:51.434971 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:51.435134 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:51.435412 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 02:22:51.435586 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":171,"threads_started":1}
I20260808 02:22:51.437192 24286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 790053ba74214d85bd7938da3c9bbe5f: No bootstrap required, opened a new log
I20260808 02:22:51.440205 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:22:51.441548 24286 raft_consensus.cc:348] T 00000000000000000000000000000000 P 790053ba74214d85bd7938da3c9bbe5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790053ba74214d85bd7938da3c9bbe5f" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 40125 } }
I20260808 02:22:51.441840 24286 raft_consensus.cc:374] T 00000000000000000000000000000000 P 790053ba74214d85bd7938da3c9bbe5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:51.441895 24286 raft_consensus.cc:729] T 00000000000000000000000000000000 P 790053ba74214d85bd7938da3c9bbe5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 790053ba74214d85bd7938da3c9bbe5f, State: Initialized, Role: FOLLOWER
I20260808 02:22:51.442435 24286 consensus_queue.cc:260] T 00000000000000000000000000000000 P 790053ba74214d85bd7938da3c9bbe5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790053ba74214d85bd7938da3c9bbe5f" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 40125 } }
I20260808 02:22:51.442674 24286 raft_consensus.cc:388] T 00000000000000000000000000000000 P 790053ba74214d85bd7938da3c9bbe5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:51.442763 24286 raft_consensus.cc:482] T 00000000000000000000000000000000 P 790053ba74214d85bd7938da3c9bbe5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:51.442885 24286 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 790053ba74214d85bd7938da3c9bbe5f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:51.444330 24286 raft_consensus.cc:504] T 00000000000000000000000000000000 P 790053ba74214d85bd7938da3c9bbe5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790053ba74214d85bd7938da3c9bbe5f" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 40125 } }
I20260808 02:22:51.444743 24286 leader_election.cc:302] T 00000000000000000000000000000000 P 790053ba74214d85bd7938da3c9bbe5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 790053ba74214d85bd7938da3c9bbe5f; no voters:
I20260808 02:22:51.445144 24286 leader_election.cc:288] T 00000000000000000000000000000000 P 790053ba74214d85bd7938da3c9bbe5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:22:51.445230 24307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 790053ba74214d85bd7938da3c9bbe5f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:51.445478 24307 raft_consensus.cc:686] T 00000000000000000000000000000000 P 790053ba74214d85bd7938da3c9bbe5f [term 1 LEADER]: Becoming Leader. State: Replica: 790053ba74214d85bd7938da3c9bbe5f, State: Running, Role: LEADER
I20260808 02:22:51.445981 24307 consensus_queue.cc:237] T 00000000000000000000000000000000 P 790053ba74214d85bd7938da3c9bbe5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790053ba74214d85bd7938da3c9bbe5f" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 40125 } }
I20260808 02:22:51.446758 24286 sys_catalog.cc:564] T 00000000000000000000000000000000 P 790053ba74214d85bd7938da3c9bbe5f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:51.447227 24291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:51.442401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58089 (local address 127.24.79.254:40125)
0808 02:22:51.442547 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:51.442553 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:51.442692 (+ 139us) server_negotiation.cc:407] Connection header received
0808 02:22:51.442857 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:51.442861 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:51.442950 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:51.443052 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:51.443326 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:51.444066 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:51.444961 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:51.445355 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:51.445475 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:51.446547 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:51.446551 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:51.446553 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:51.446570 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:51.446607 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:51.446633 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:51.446725 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:51.447041 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 02:22:51.447087 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 02:22:51.448199 24310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 790053ba74214d85bd7938da3c9bbe5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 790053ba74214d85bd7938da3c9bbe5f. Latest consensus state: current_term: 1 leader_uuid: "790053ba74214d85bd7938da3c9bbe5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790053ba74214d85bd7938da3c9bbe5f" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 40125 } } }
I20260808 02:22:51.448348 24310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 790053ba74214d85bd7938da3c9bbe5f [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:51.448696 24308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 790053ba74214d85bd7938da3c9bbe5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "790053ba74214d85bd7938da3c9bbe5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790053ba74214d85bd7938da3c9bbe5f" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 40125 } } }
I20260808 02:22:51.448784 24308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 790053ba74214d85bd7938da3c9bbe5f [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:51.449141 24319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:22:51.453083 24319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:22:51.459425 24319 catalog_manager.cc:1347] Generated new cluster ID: e8f57b0ad9924fc586c2e056d2f0e5ec
I20260808 02:22:51.459532 24319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:22:51.471359 24319 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:22:51.472621 24319 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:22:51.483892 24319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 790053ba74214d85bd7938da3c9bbe5f: Generated new TSK 0
I20260808 02:22:51.485008 24319 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:22:51.495395 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 24186
W20260808 02:22:51.514668 24165 connection.cc:535] client connection to 127.24.79.254:40125 recv error: Network error: failed to read from TLS socket (remote: 127.24.79.254:40125): Connection reset by peer (error 104)
2026-08-08T02:22:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1642 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:22:51 dist-test-slave-1vmg krb5kdc[24337](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:22:51 dist-test-slave-1vmg krb5kdc[24337](info): set up 2 sockets
Aug 08 02:22:51 dist-test-slave-1vmg krb5kdc[24337](info): commencing operation
krb5kdc: starting...
W20260808 02:22:52.667397 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:22:52 dist-test-slave-1vmg krb5kdc[24337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:22:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:22:52.811524 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:46087
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:42345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:46087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:52.934000 24358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:52.934336 24358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:52.934405 24358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:52.938727 24358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:52.938829 24358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:52.938858 24358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:52.938879 24358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:52.938899 24358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:52.944742 24358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:42345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:46087
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:46087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:52.946017 24358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:52.947171 24358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:22:52.956491 24358 server_base.cc:1034] running on GCE node
W20260808 02:22:52.956485 24369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:52.956516 24367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:52.956473 24372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:52.957559 24358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:52.958853 24358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:52.960045 24358 hybrid_clock.cc:648] HybridClock initialized: now 1786155772960000 us; error 62 us; skew 500 ppm
Aug 08 02:22:52 dist-test-slave-1vmg krb5kdc[24337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155772, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:52.963121 24358 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:22:52.964221 24358 webserver.cc:460] Webserver started at http://127.24.79.254:32883/ using document root <none> and password file <none>
I20260808 02:22:52.964848 24358 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:52.964923 24358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:52.965157 24358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:52.967581 24358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "6c0b2eabd6654f048bff375f7a86abe0"
format_stamp: "Formatted at 2026-08-08 02:22:52 on dist-test-slave-1vmg"
server_key: "8878fa2e5ceacca40379447fcd293903"
server_key_iv: "578d37163e84352bab25f126fa561f00"
server_key_version: "encryptionkey@0"
I20260808 02:22:52.968230 24358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "6c0b2eabd6654f048bff375f7a86abe0"
format_stamp: "Formatted at 2026-08-08 02:22:52 on dist-test-slave-1vmg"
server_key: "8878fa2e5ceacca40379447fcd293903"
server_key_iv: "578d37163e84352bab25f126fa561f00"
server_key_version: "encryptionkey@0"
I20260808 02:22:52.972589 24358 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:22:52.975544 24381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:52.977085 24358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:22:52.977252 24358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "6c0b2eabd6654f048bff375f7a86abe0"
format_stamp: "Formatted at 2026-08-08 02:22:52 on dist-test-slave-1vmg"
server_key: "8878fa2e5ceacca40379447fcd293903"
server_key_iv: "578d37163e84352bab25f126fa561f00"
server_key_version: "encryptionkey@0"
I20260808 02:22:52.977387 24358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:52.982549 24358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:52.986370 24358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:52.986624 24358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:52.994900 24358 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:46087
I20260808 02:22:52.995112 24460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:46087 every 8 connection(s)
I20260808 02:22:52.996182 24358 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:52.999791 24465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:53.000803 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 24358
I20260808 02:22:53.000944 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:53.001246 24895 external_mini_cluster.cc:1428] Setting key a252d00476c0e68e29536e55e7031329
I20260808 02:22:53.007597 24465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c0b2eabd6654f048bff375f7a86abe0: Bootstrap starting.
I20260808 02:22:53.008041 24471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:53.003064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42305 (local address 127.24.79.254:46087)
0808 02:22:53.003605 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:53.003628 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:53.003678 (+ 50us) server_negotiation.cc:407] Connection header received
0808 02:22:53.004582 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:53.004610 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:53.005031 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:53.005540 (+ 509us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:53.006402 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:53.006409 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:53.006428 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:53.006441 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:53.006521 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:53.006549 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:53.006707 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:53.006783 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 02:22:53.006987 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":131,"threads_started":1}
I20260808 02:22:53.010418 24465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c0b2eabd6654f048bff375f7a86abe0: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:53.011584 24465 log.cc:824] T 00000000000000000000000000000000 P 6c0b2eabd6654f048bff375f7a86abe0: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:53.011744 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:22:53.014425 24465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c0b2eabd6654f048bff375f7a86abe0: No bootstrap required, opened a new log
I20260808 02:22:53.016084 24471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:53.014109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53527 (local address 127.24.79.254:46087)
0808 02:22:53.014257 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:53.014262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:53.014513 (+ 251us) server_negotiation.cc:407] Connection header received
0808 02:22:53.014716 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:53.014719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:53.014763 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:53.014844 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:53.015687 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:53.015690 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:53.015692 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:53.015704 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:53.015735 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:53.015746 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:53.015825 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:53.015926 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 02:22:53.015961 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 02:22:53.019409 24465 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c0b2eabd6654f048bff375f7a86abe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c0b2eabd6654f048bff375f7a86abe0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46087 } }
I20260808 02:22:53.019796 24465 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c0b2eabd6654f048bff375f7a86abe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:53.019932 24465 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c0b2eabd6654f048bff375f7a86abe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c0b2eabd6654f048bff375f7a86abe0, State: Initialized, Role: FOLLOWER
I20260808 02:22:53.020462 24465 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c0b2eabd6654f048bff375f7a86abe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c0b2eabd6654f048bff375f7a86abe0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46087 } }
I20260808 02:22:53.020620 24465 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c0b2eabd6654f048bff375f7a86abe0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:53.020669 24465 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c0b2eabd6654f048bff375f7a86abe0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:53.020763 24465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c0b2eabd6654f048bff375f7a86abe0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:53.022467 24465 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c0b2eabd6654f048bff375f7a86abe0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c0b2eabd6654f048bff375f7a86abe0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46087 } }
I20260808 02:22:53.022986 24465 leader_election.cc:302] T 00000000000000000000000000000000 P 6c0b2eabd6654f048bff375f7a86abe0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c0b2eabd6654f048bff375f7a86abe0; no voters:
I20260808 02:22:53.023597 24465 leader_election.cc:288] T 00000000000000000000000000000000 P 6c0b2eabd6654f048bff375f7a86abe0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:22:53.024029 24479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c0b2eabd6654f048bff375f7a86abe0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:53.024426 24479 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c0b2eabd6654f048bff375f7a86abe0 [term 1 LEADER]: Becoming Leader. State: Replica: 6c0b2eabd6654f048bff375f7a86abe0, State: Running, Role: LEADER
I20260808 02:22:53.024812 24479 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c0b2eabd6654f048bff375f7a86abe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c0b2eabd6654f048bff375f7a86abe0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46087 } }
I20260808 02:22:53.025312 24465 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c0b2eabd6654f048bff375f7a86abe0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:53.026822 24480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c0b2eabd6654f048bff375f7a86abe0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c0b2eabd6654f048bff375f7a86abe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c0b2eabd6654f048bff375f7a86abe0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46087 } } }
I20260808 02:22:53.026873 24482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c0b2eabd6654f048bff375f7a86abe0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c0b2eabd6654f048bff375f7a86abe0. Latest consensus state: current_term: 1 leader_uuid: "6c0b2eabd6654f048bff375f7a86abe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c0b2eabd6654f048bff375f7a86abe0" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 46087 } } }
I20260808 02:22:53.027037 24482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c0b2eabd6654f048bff375f7a86abe0 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:53.027036 24480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c0b2eabd6654f048bff375f7a86abe0 [sys.catalog]: This master's current role is: LEADER
W20260808 02:22:53.031574 24502 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6c0b2eabd6654f048bff375f7a86abe0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:22:53.031684 24502 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:22:53.031800 24490 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:22:53.032910 24490 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:22:53.040509 24490 catalog_manager.cc:1347] Generated new cluster ID: 287c76b4c53842488128426eb76fa9cd
I20260808 02:22:53.040622 24490 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:22:53.062305 24490 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:22:53.062709 24490 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:22:53.069110 24490 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c0b2eabd6654f048bff375f7a86abe0: Generated new TSK 0
I20260808 02:22:53.069967 24490 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:22:53.082952 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 24358
2026-08-08T02:22:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1581 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:22:53 dist-test-slave-1vmg krb5kdc[24512](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:22:53 dist-test-slave-1vmg krb5kdc[24512](info): set up 2 sockets
Aug 08 02:22:53 dist-test-slave-1vmg krb5kdc[24512](info): commencing operation
krb5kdc: starting...
W20260808 02:22:54.248847 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:22:54 dist-test-slave-1vmg krb5kdc[24512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:22:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:22:54.395463 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:35045
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:40681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:35045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:54.519577 24538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:54.519975 24538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:54.520044 24538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:54.524497 24538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:54.524595 24538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:54.524626 24538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:54.524648 24538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:54.524672 24538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:54.530637 24538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:35045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:35045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:54.531970 24538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:54.533097 24538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:54.542956 24549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:54.542964 24548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:54.543305 24538 server_base.cc:1034] running on GCE node
W20260808 02:22:54.542948 24552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:54.544165 24538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:54.545514 24538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:54.546728 24538 hybrid_clock.cc:648] HybridClock initialized: now 1786155774546698 us; error 47 us; skew 500 ppm
Aug 08 02:22:54 dist-test-slave-1vmg krb5kdc[24512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155774, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:54.550236 24538 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:22:54.551642 24538 webserver.cc:460] Webserver started at http://127.24.79.254:32923/ using document root <none> and password file <none>
I20260808 02:22:54.552341 24538 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:54.552429 24538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:54.552691 24538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:54.555295 24538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "c3a818b9badd49858637940a6fe5748f"
format_stamp: "Formatted at 2026-08-08 02:22:54 on dist-test-slave-1vmg"
server_key: "a6a6f58f6a4f3b620f71860723988df9"
server_key_iv: "e0717643bd159d44de0ff083871df531"
server_key_version: "encryptionkey@0"
I20260808 02:22:54.556023 24538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "c3a818b9badd49858637940a6fe5748f"
format_stamp: "Formatted at 2026-08-08 02:22:54 on dist-test-slave-1vmg"
server_key: "a6a6f58f6a4f3b620f71860723988df9"
server_key_iv: "e0717643bd159d44de0ff083871df531"
server_key_version: "encryptionkey@0"
I20260808 02:22:54.560701 24538 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:22:54.563788 24561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:54.565335 24538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:22:54.565496 24538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "c3a818b9badd49858637940a6fe5748f"
format_stamp: "Formatted at 2026-08-08 02:22:54 on dist-test-slave-1vmg"
server_key: "a6a6f58f6a4f3b620f71860723988df9"
server_key_iv: "e0717643bd159d44de0ff083871df531"
server_key_version: "encryptionkey@0"
I20260808 02:22:54.565625 24538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:54.580921 24538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:54.584789 24538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:54.585009 24538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:54.593204 24538 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:35045
I20260808 02:22:54.593276 24647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:35045 every 8 connection(s)
I20260808 02:22:54.594377 24538 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:54.594805 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 24538
I20260808 02:22:54.594992 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:54.595316 24895 external_mini_cluster.cc:1428] Setting key 8c8cdfa540651148255bac2d09b2a7d3
I20260808 02:22:54.598109 24648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 02:22:54 dist-test-slave-1vmg krb5kdc[24512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:22:54.605700 24648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3a818b9badd49858637940a6fe5748f: Bootstrap starting.
I20260808 02:22:54.608363 24648 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3a818b9badd49858637940a6fe5748f: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:54.609242 24648 log.cc:824] T 00000000000000000000000000000000 P c3a818b9badd49858637940a6fe5748f: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:54.611413 24657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:54.596926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58521 (local address 127.24.79.254:35045)
0808 02:22:54.597404 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:54.597423 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:54.597465 (+ 42us) server_negotiation.cc:407] Connection header received
0808 02:22:54.598179 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:54.598200 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:54.598627 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:54.599025 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:54.600046 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:54.601169 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:54.602395 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:54.602908 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:54.603114 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:54.606138 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:54.606144 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:54.606161 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:54.606197 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:54.608037 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:54.608612 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:54.608618 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:54.608624 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:54.608768 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:54.609077 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:54.609081 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:54.609082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:54.609615 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:54.609823 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:54.610167 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 02:22:54.610513 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":126,"threads_started":1}
I20260808 02:22:54.611680 24648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3a818b9badd49858637940a6fe5748f: No bootstrap required, opened a new log
I20260808 02:22:54.614713 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:22:54.615931 24648 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3a818b9badd49858637940a6fe5748f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3a818b9badd49858637940a6fe5748f" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 35045 } }
I20260808 02:22:54.616308 24648 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3a818b9badd49858637940a6fe5748f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:54.616389 24648 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3a818b9badd49858637940a6fe5748f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3a818b9badd49858637940a6fe5748f, State: Initialized, Role: FOLLOWER
I20260808 02:22:54.616943 24648 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3a818b9badd49858637940a6fe5748f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3a818b9badd49858637940a6fe5748f" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 35045 } }
I20260808 02:22:54.617098 24648 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3a818b9badd49858637940a6fe5748f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:54.617184 24648 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3a818b9badd49858637940a6fe5748f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:54.617286 24648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3a818b9badd49858637940a6fe5748f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:54.618682 24648 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3a818b9badd49858637940a6fe5748f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3a818b9badd49858637940a6fe5748f" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 35045 } }
I20260808 02:22:54.619210 24648 leader_election.cc:302] T 00000000000000000000000000000000 P c3a818b9badd49858637940a6fe5748f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3a818b9badd49858637940a6fe5748f; no voters:
I20260808 02:22:54.619676 24648 leader_election.cc:288] T 00000000000000000000000000000000 P c3a818b9badd49858637940a6fe5748f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:22:54.619810 24668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3a818b9badd49858637940a6fe5748f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:54.620054 24668 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3a818b9badd49858637940a6fe5748f [term 1 LEADER]: Becoming Leader. State: Replica: c3a818b9badd49858637940a6fe5748f, State: Running, Role: LEADER
I20260808 02:22:54.620440 24668 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3a818b9badd49858637940a6fe5748f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3a818b9badd49858637940a6fe5748f" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 35045 } }
I20260808 02:22:54.621268 24648 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3a818b9badd49858637940a6fe5748f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:54.623925 24669 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3a818b9badd49858637940a6fe5748f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3a818b9badd49858637940a6fe5748f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3a818b9badd49858637940a6fe5748f" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 35045 } } }
I20260808 02:22:54.624089 24669 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3a818b9badd49858637940a6fe5748f [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:54.624372 24671 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3a818b9badd49858637940a6fe5748f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3a818b9badd49858637940a6fe5748f. Latest consensus state: current_term: 1 leader_uuid: "c3a818b9badd49858637940a6fe5748f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3a818b9badd49858637940a6fe5748f" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 35045 } } }
I20260808 02:22:54.624480 24671 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3a818b9badd49858637940a6fe5748f [sys.catalog]: This master's current role is: LEADER
W20260808 02:22:54.627315 24687 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c3a818b9badd49858637940a6fe5748f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:22:54.627395 24687 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:22:54.627492 24688 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:22:54.628211 24688 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:22:54.628840 24657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:54.617321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45733 (local address 127.24.79.254:35045)
0808 02:22:54.617473 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:54.617478 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:54.617770 (+ 292us) server_negotiation.cc:407] Connection header received
0808 02:22:54.617884 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:54.617887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:54.617938 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:54.618051 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:54.618987 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:54.619749 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:54.620938 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:54.621383 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:54.621509 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:54.623470 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:54.623474 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:54.623477 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:54.623508 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:54.626208 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:54.626812 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:54.626816 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:54.626818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:54.626953 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:54.627322 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:54.627326 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:54.627328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:54.627602 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:54.627741 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:54.628523 (+ 782us) server_negotiation.cc:299] Negotiation successful
0808 02:22:54.628674 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:22:54.635090 24688 catalog_manager.cc:1347] Generated new cluster ID: 6bbc5e3eca2d42c096bea94a0dc5694b
I20260808 02:22:54.635198 24688 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:22:54.653689 24688 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:22:54.655224 24688 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:22:54.665524 24688 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3a818b9badd49858637940a6fe5748f: Generated new TSK 0
I20260808 02:22:54.666523 24688 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:22:54.679445 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 24538
2026-08-08T02:22:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1599 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:22:54 dist-test-slave-1vmg krb5kdc[24697](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:22:54 dist-test-slave-1vmg krb5kdc[24697](info): set up 2 sockets
Aug 08 02:22:54 dist-test-slave-1vmg krb5kdc[24697](info): commencing operation
krb5kdc: starting...
W20260808 02:22:55.830906 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:22:55 dist-test-slave-1vmg krb5kdc[24697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:22:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:22:55.982458 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:41489
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:44753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:41489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:56.112629 24713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:56.112983 24713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:56.113049 24713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:56.117449 24713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:56.117552 24713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:56.117581 24713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:56.117605 24713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:56.117628 24713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:56.124436 24713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:44753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:41489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:41489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:56.125806 24713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:56.127182 24713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:56.138597 24726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:56.138597 24723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:56.138609 24722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:56.139029 24713 server_base.cc:1034] running on GCE node
I20260808 02:22:56.139810 24713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:56.141217 24713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:56.142459 24713 hybrid_clock.cc:648] HybridClock initialized: now 1786155776142404 us; error 79 us; skew 500 ppm
Aug 08 02:22:56 dist-test-slave-1vmg krb5kdc[24697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155776, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:56.151908 24713 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:22:56.153213 24713 webserver.cc:460] Webserver started at http://127.24.79.254:35079/ using document root <none> and password file <none>
I20260808 02:22:56.153913 24713 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:56.154002 24713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:56.154263 24713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:56.157073 24713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "232e19be86474181b7ba02726e59d4e8"
format_stamp: "Formatted at 2026-08-08 02:22:56 on dist-test-slave-1vmg"
server_key: "a8ab6ba6ea259d94665119e967e09265"
server_key_iv: "31652f29b9c5373dac996e32048bfd24"
server_key_version: "encryptionkey@0"
I20260808 02:22:56.157805 24713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "232e19be86474181b7ba02726e59d4e8"
format_stamp: "Formatted at 2026-08-08 02:22:56 on dist-test-slave-1vmg"
server_key: "a8ab6ba6ea259d94665119e967e09265"
server_key_iv: "31652f29b9c5373dac996e32048bfd24"
server_key_version: "encryptionkey@0"
I20260808 02:22:56.162639 24713 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 02:22:56.165832 24737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:56.167572 24713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:22:56.167755 24713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "232e19be86474181b7ba02726e59d4e8"
format_stamp: "Formatted at 2026-08-08 02:22:56 on dist-test-slave-1vmg"
server_key: "a8ab6ba6ea259d94665119e967e09265"
server_key_iv: "31652f29b9c5373dac996e32048bfd24"
server_key_version: "encryptionkey@0"
I20260808 02:22:56.167897 24713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:56.183679 24713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:56.187883 24713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:56.188141 24713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:56.196717 24713 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:41489
I20260808 02:22:56.196821 24830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:41489 every 8 connection(s)
I20260808 02:22:56.197976 24713 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:56.201774 24831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:56.202298 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 24713
I20260808 02:22:56.202461 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:56.202793 24895 external_mini_cluster.cc:1428] Setting key 8281418cc00fb7be4c7b33c34dcab84f
I20260808 02:22:56.210155 24831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 232e19be86474181b7ba02726e59d4e8: Bootstrap starting.
I20260808 02:22:56.213107 24831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 232e19be86474181b7ba02726e59d4e8: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:56.213131 24835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:56.204480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41571 (local address 127.24.79.254:41489)
0808 02:22:56.204956 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:56.204974 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:56.205015 (+ 41us) server_negotiation.cc:407] Connection header received
0808 02:22:56.205785 (+ 770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:56.205808 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:56.206210 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:56.206531 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:56.207014 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:56.208115 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:56.209411 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:56.209959 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:56.210152 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:56.211094 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:56.211099 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:56.211115 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:56.211561 (+ 446us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:56.211633 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:56.211698 (+ 65us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:56.211782 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:56.211985 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 02:22:56.212246 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":137,"threads_started":1}
I20260808 02:22:56.214010 24831 log.cc:824] T 00000000000000000000000000000000 P 232e19be86474181b7ba02726e59d4e8: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:56.216391 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:22:56.216437 24831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 232e19be86474181b7ba02726e59d4e8: No bootstrap required, opened a new log
I20260808 02:22:56.221374 24831 raft_consensus.cc:348] T 00000000000000000000000000000000 P 232e19be86474181b7ba02726e59d4e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "232e19be86474181b7ba02726e59d4e8" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41489 } }
I20260808 02:22:56.221781 24831 raft_consensus.cc:374] T 00000000000000000000000000000000 P 232e19be86474181b7ba02726e59d4e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:56.221863 24831 raft_consensus.cc:729] T 00000000000000000000000000000000 P 232e19be86474181b7ba02726e59d4e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 232e19be86474181b7ba02726e59d4e8, State: Initialized, Role: FOLLOWER
I20260808 02:22:56.222506 24831 consensus_queue.cc:260] T 00000000000000000000000000000000 P 232e19be86474181b7ba02726e59d4e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "232e19be86474181b7ba02726e59d4e8" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41489 } }
I20260808 02:22:56.222677 24831 raft_consensus.cc:388] T 00000000000000000000000000000000 P 232e19be86474181b7ba02726e59d4e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:56.222733 24831 raft_consensus.cc:482] T 00000000000000000000000000000000 P 232e19be86474181b7ba02726e59d4e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:56.222819 24831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 232e19be86474181b7ba02726e59d4e8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:56.223946 24835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:56.218877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45435 (local address 127.24.79.254:41489)
0808 02:22:56.219077 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:56.219082 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:56.219374 (+ 292us) server_negotiation.cc:407] Connection header received
0808 02:22:56.219497 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:56.219499 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:56.219549 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:56.219668 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:56.220074 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:56.220659 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:56.221854 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:56.222234 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:56.222351 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:56.223486 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:56.223490 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:56.223492 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:56.223509 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:56.223547 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:56.223560 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:56.223636 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:56.223727 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 02:22:56.223806 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260808 02:22:56.224417 24831 raft_consensus.cc:504] T 00000000000000000000000000000000 P 232e19be86474181b7ba02726e59d4e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "232e19be86474181b7ba02726e59d4e8" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41489 } }
I20260808 02:22:56.224830 24831 leader_election.cc:302] T 00000000000000000000000000000000 P 232e19be86474181b7ba02726e59d4e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 232e19be86474181b7ba02726e59d4e8; no voters:
I20260808 02:22:56.225224 24831 leader_election.cc:288] T 00000000000000000000000000000000 P 232e19be86474181b7ba02726e59d4e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:22:56.225334 24846 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 232e19be86474181b7ba02726e59d4e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:56.225613 24846 raft_consensus.cc:686] T 00000000000000000000000000000000 P 232e19be86474181b7ba02726e59d4e8 [term 1 LEADER]: Becoming Leader. State: Replica: 232e19be86474181b7ba02726e59d4e8, State: Running, Role: LEADER
I20260808 02:22:56.225970 24846 consensus_queue.cc:237] T 00000000000000000000000000000000 P 232e19be86474181b7ba02726e59d4e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "232e19be86474181b7ba02726e59d4e8" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41489 } }
I20260808 02:22:56.226770 24831 sys_catalog.cc:564] T 00000000000000000000000000000000 P 232e19be86474181b7ba02726e59d4e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:56.227895 24850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 232e19be86474181b7ba02726e59d4e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "232e19be86474181b7ba02726e59d4e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "232e19be86474181b7ba02726e59d4e8" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41489 } } }
I20260808 02:22:56.227928 24853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 232e19be86474181b7ba02726e59d4e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 232e19be86474181b7ba02726e59d4e8. Latest consensus state: current_term: 1 leader_uuid: "232e19be86474181b7ba02726e59d4e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "232e19be86474181b7ba02726e59d4e8" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41489 } } }
I20260808 02:22:56.228145 24853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 232e19be86474181b7ba02726e59d4e8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:56.228175 24850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 232e19be86474181b7ba02726e59d4e8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:56.228590 24862 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:22:56.232605 24862 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:22:56.239120 24862 catalog_manager.cc:1347] Generated new cluster ID: f62b77190a404d7b851cd3647af68a15
I20260808 02:22:56.239235 24862 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:22:56.256855 24862 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:22:56.257980 24862 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:22:56.265378 24862 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 232e19be86474181b7ba02726e59d4e8: Generated new TSK 0
I20260808 02:22:56.266248 24862 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:22:56.273828 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 24713
2026-08-08T02:22:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1589 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:22:56 dist-test-slave-1vmg krb5kdc[24881](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:22:56 dist-test-slave-1vmg krb5kdc[24881](info): set up 2 sockets
Aug 08 02:22:56 dist-test-slave-1vmg krb5kdc[24881](info): commencing operation
krb5kdc: starting...
W20260808 02:22:57.448601 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:22:57 dist-test-slave-1vmg krb5kdc[24881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:22:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:22:57.600378 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:32895
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:38321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:32895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:57.726253 24908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:57.726625 24908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:57.726697 24908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:57.731276 24908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:57.731387 24908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:57.731420 24908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:57.731449 24908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:57.731487 24908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:57.738097 24908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:38321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:32895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:32895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:57.739540 24908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:57.740749 24908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:57.750376 24915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:57.750365 24913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:57.750362 24917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:57.750702 24908 server_base.cc:1034] running on GCE node
I20260808 02:22:57.751371 24908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:57.752653 24908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:57.753844 24908 hybrid_clock.cc:648] HybridClock initialized: now 1786155777753823 us; error 37 us; skew 500 ppm
Aug 08 02:22:57 dist-test-slave-1vmg krb5kdc[24881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155777, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:57.757395 24908 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:22:57.758682 24908 webserver.cc:460] Webserver started at http://127.24.79.254:38725/ using document root <none> and password file <none>
I20260808 02:22:57.759394 24908 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:57.759479 24908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:57.759732 24908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:57.762287 24908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "1234a20bb5504d27b1cde8e6d9c8d31e"
format_stamp: "Formatted at 2026-08-08 02:22:57 on dist-test-slave-1vmg"
server_key: "cd4261ab510c1de597f5041a3377f18a"
server_key_iv: "7268f2f49d3b220b5635b4f8b10ee680"
server_key_version: "encryptionkey@0"
I20260808 02:22:57.763037 24908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "1234a20bb5504d27b1cde8e6d9c8d31e"
format_stamp: "Formatted at 2026-08-08 02:22:57 on dist-test-slave-1vmg"
server_key: "cd4261ab510c1de597f5041a3377f18a"
server_key_iv: "7268f2f49d3b220b5635b4f8b10ee680"
server_key_version: "encryptionkey@0"
I20260808 02:22:57.767580 24908 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:22:57.770534 24928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:57.772221 24908 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:22:57.772392 24908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "1234a20bb5504d27b1cde8e6d9c8d31e"
format_stamp: "Formatted at 2026-08-08 02:22:57 on dist-test-slave-1vmg"
server_key: "cd4261ab510c1de597f5041a3377f18a"
server_key_iv: "7268f2f49d3b220b5635b4f8b10ee680"
server_key_version: "encryptionkey@0"
I20260808 02:22:57.772518 24908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:57.778046 24908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:57.782080 24908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:57.782311 24908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:57.790000 24908 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:32895
I20260808 02:22:57.790031 25012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:32895 every 8 connection(s)
I20260808 02:22:57.791189 24908 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:57.794663 25013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:57.799512 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 24908
I20260808 02:22:57.799656 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:57.799968 24895 external_mini_cluster.cc:1428] Setting key e7684b817b2637cfbddf2e30195ddba0
I20260808 02:22:57.801796 25013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1234a20bb5504d27b1cde8e6d9c8d31e: Bootstrap starting.
I20260808 02:22:57.805075 25013 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1234a20bb5504d27b1cde8e6d9c8d31e: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:57.806157 25013 log.cc:824] T 00000000000000000000000000000000 P 1234a20bb5504d27b1cde8e6d9c8d31e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:57.806711 25016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:57.801767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35865 (local address 127.24.79.254:32895)
0808 02:22:57.802260 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:57.802284 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:57.802336 (+ 52us) server_negotiation.cc:407] Connection header received
0808 02:22:57.803386 (+ 1050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:57.803417 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:57.803860 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:57.804302 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:57.805100 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:57.805108 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:57.805126 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:57.805141 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:57.805219 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:57.805253 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:57.805388 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:57.805508 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 02:22:57.805692 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":140,"threads_started":1}
I20260808 02:22:57.808724 25013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1234a20bb5504d27b1cde8e6d9c8d31e: No bootstrap required, opened a new log
I20260808 02:22:57.809966 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:22:57.813195 25013 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1234a20bb5504d27b1cde8e6d9c8d31e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1234a20bb5504d27b1cde8e6d9c8d31e" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 32895 } }
I20260808 02:22:57.813547 25013 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1234a20bb5504d27b1cde8e6d9c8d31e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:22:57.813628 25013 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1234a20bb5504d27b1cde8e6d9c8d31e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234a20bb5504d27b1cde8e6d9c8d31e, State: Initialized, Role: FOLLOWER
I20260808 02:22:57.813954 25016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:57.812202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40927 (local address 127.24.79.254:32895)
0808 02:22:57.812348 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:57.812354 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:57.812441 (+ 87us) server_negotiation.cc:407] Connection header received
0808 02:22:57.812558 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:57.812561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:57.812605 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:57.812717 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:57.813410 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:57.813416 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:57.813419 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:22:57.813435 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:57.813472 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:22:57.813489 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:57.813609 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:57.813773 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 02:22:57.813819 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 02:22:57.814226 25013 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1234a20bb5504d27b1cde8e6d9c8d31e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1234a20bb5504d27b1cde8e6d9c8d31e" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 32895 } }
I20260808 02:22:57.814431 25013 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1234a20bb5504d27b1cde8e6d9c8d31e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:22:57.814517 25013 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1234a20bb5504d27b1cde8e6d9c8d31e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:22:57.814641 25013 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1234a20bb5504d27b1cde8e6d9c8d31e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:22:57.815882 25013 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1234a20bb5504d27b1cde8e6d9c8d31e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1234a20bb5504d27b1cde8e6d9c8d31e" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 32895 } }
I20260808 02:22:57.816303 25013 leader_election.cc:302] T 00000000000000000000000000000000 P 1234a20bb5504d27b1cde8e6d9c8d31e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1234a20bb5504d27b1cde8e6d9c8d31e; no voters:
I20260808 02:22:57.816718 25013 leader_election.cc:288] T 00000000000000000000000000000000 P 1234a20bb5504d27b1cde8e6d9c8d31e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:22:57.816852 25025 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1234a20bb5504d27b1cde8e6d9c8d31e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:22:57.817092 25025 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1234a20bb5504d27b1cde8e6d9c8d31e [term 1 LEADER]: Becoming Leader. State: Replica: 1234a20bb5504d27b1cde8e6d9c8d31e, State: Running, Role: LEADER
I20260808 02:22:57.817493 25025 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1234a20bb5504d27b1cde8e6d9c8d31e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1234a20bb5504d27b1cde8e6d9c8d31e" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 32895 } }
I20260808 02:22:57.818392 25013 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1234a20bb5504d27b1cde8e6d9c8d31e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:22:57.819422 25027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1234a20bb5504d27b1cde8e6d9c8d31e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1234a20bb5504d27b1cde8e6d9c8d31e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1234a20bb5504d27b1cde8e6d9c8d31e" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 32895 } } }
I20260808 02:22:57.819588 25027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1234a20bb5504d27b1cde8e6d9c8d31e [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:57.819969 25028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1234a20bb5504d27b1cde8e6d9c8d31e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1234a20bb5504d27b1cde8e6d9c8d31e. Latest consensus state: current_term: 1 leader_uuid: "1234a20bb5504d27b1cde8e6d9c8d31e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1234a20bb5504d27b1cde8e6d9c8d31e" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 32895 } } }
I20260808 02:22:57.820075 25028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1234a20bb5504d27b1cde8e6d9c8d31e [sys.catalog]: This master's current role is: LEADER
I20260808 02:22:57.820978 25038 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:22:57.824311 25038 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:22:57.830942 25038 catalog_manager.cc:1347] Generated new cluster ID: c636ca2f5fc8456aaa23fe7f89a5c47f
I20260808 02:22:57.831048 25038 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:22:57.839471 25038 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:22:57.839990 25038 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:22:57.848302 25038 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1234a20bb5504d27b1cde8e6d9c8d31e: Generated new TSK 0
I20260808 02:22:57.849308 25038 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:22:57.859907 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 24908
2026-08-08T02:22:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1594 ms)
[----------] 18 tests from AuthTokenIssuingTest (28924 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:22:57 dist-test-slave-1vmg krb5kdc[25054](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:22:57 dist-test-slave-1vmg krb5kdc[25054](info): set up 2 sockets
Aug 08 02:22:57 dist-test-slave-1vmg krb5kdc[25054](info): commencing operation
krb5kdc: starting...
W20260808 02:22:59.013247 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:22:59 dist-test-slave-1vmg krb5kdc[25054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:22:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:22:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:22:59.162985 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:37055
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:44173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:37055,127.24.79.253:36141,127.24.79.252:37011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:22:59.290521 25081 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 02:22:59.291006 25081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:59.291061 25081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:59.291137 25081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:59.295495 25081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:59.295579 25081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:59.295598 25081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:59.295614 25081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:59.295637 25081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:59.301630 25081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:44173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:37055,127.24.79.253:36141,127.24.79.252:37011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:37055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:59.302882 25081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:59.304011 25081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:59.313396 25087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:59.313396 25086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:59.313395 25090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:59.313856 25081 server_base.cc:1034] running on GCE node
I20260808 02:22:59.314548 25081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:59.315873 25081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:59.317147 25081 hybrid_clock.cc:648] HybridClock initialized: now 1786155779317111 us; error 48 us; skew 500 ppm
Aug 08 02:22:59 dist-test-slave-1vmg krb5kdc[25054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155779, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:59.320911 25081 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:22:59.322109 25081 webserver.cc:460] Webserver started at http://127.24.79.254:33249/ using document root <none> and password file <none>
I20260808 02:22:59.322793 25081 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:59.322865 25081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:59.323120 25081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:59.325451 25081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "3e73934c54a84eafb35b93675ab9a92c"
format_stamp: "Formatted at 2026-08-08 02:22:59 on dist-test-slave-1vmg"
server_key: "82e725b208d19adbd50fe08568e2d248"
server_key_iv: "72cc6c4fbc1c9ace34f5672a0d269281"
server_key_version: "encryptionkey@0"
I20260808 02:22:59.326110 25081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "3e73934c54a84eafb35b93675ab9a92c"
format_stamp: "Formatted at 2026-08-08 02:22:59 on dist-test-slave-1vmg"
server_key: "82e725b208d19adbd50fe08568e2d248"
server_key_iv: "72cc6c4fbc1c9ace34f5672a0d269281"
server_key_version: "encryptionkey@0"
I20260808 02:22:59.331830 25081 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 02:22:59.334733 25100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:59.336408 25081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:22:59.336570 25081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "3e73934c54a84eafb35b93675ab9a92c"
format_stamp: "Formatted at 2026-08-08 02:22:59 on dist-test-slave-1vmg"
server_key: "82e725b208d19adbd50fe08568e2d248"
server_key_iv: "72cc6c4fbc1c9ace34f5672a0d269281"
server_key_version: "encryptionkey@0"
I20260808 02:22:59.336720 25081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:59.355927 25081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:59.360062 25081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:59.360319 25081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:59.368280 25081 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:37055
I20260808 02:22:59.368561 25191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:37055 every 8 connection(s)
I20260808 02:22:59.369410 25081 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:22:59.372473 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 25081
I20260808 02:22:59.372613 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:22:59.372720 25192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:59.372964 24895 external_mini_cluster.cc:1428] Setting key a8cd0f9822fbb0f1ff25caaf42c8f862
I20260808 02:22:59.377368 25192 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37055 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.79.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.253@KRBTEST.COM" created.
Aug 08 02:22:59 dist-test-slave-1vmg krb5kdc[25054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155779, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:22:59.395346 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.380006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:37055 (local address 127.0.0.1:58436)
0808 02:22:59.380716 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.380759 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:59.381711 (+ 952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:59.382596 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:59.382604 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:59.383067 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:59.383396 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:59.383417 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.384673 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.384677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:59.385500 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:59.385512 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.386395 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.386399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:59.386520 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:59.387271 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:22:59.387294 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:59.389325 (+ 2031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:59.391803 (+ 2478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:59.391818 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:59.391832 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:59.392188 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:59.392675 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:59.392678 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:59.392680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:59.392820 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:59.393712 (+ 892us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:59.393718 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:59.393868 (+ 150us) client_negotiation.cc:769] Sending connection context
0808 02:22:59.394033 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 02:22:59.394371 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"spinlock_wait_cycles":50048,"thread_start_us":211,"threads_started":1}
I20260808 02:22:59.395346 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.380005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58436 (local address 127.24.79.254:37055)
0808 02:22:59.380556 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:59.380579 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:59.380885 (+ 306us) server_negotiation.cc:407] Connection header received
0808 02:22:59.382083 (+ 1198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:59.382088 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:59.382306 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:59.382506 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:59.383591 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.384532 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.385654 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.386138 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.386294 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:59.389523 (+ 3229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:59.389532 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:59.389551 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:59.389584 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:59.391630 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:59.392347 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:59.392355 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:59.392357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:59.392532 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:59.392924 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:59.392928 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:59.392930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:59.393579 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:59.393739 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:59.394197 (+ 458us) server_negotiation.cc:299] Negotiation successful
0808 02:22:59.394529 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":347,"thread_start_us":160,"threads_started":1}
I20260808 02:22:59.398255 25192 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:22:59.399179 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.398829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:36141 (local address 127.0.0.1:56776)
0808 02:22:59.399005 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.399099 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.253:36141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 02:22:59.399526 25104 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.79.253:36141: connect: Connection refused (error 111)
I20260808 02:22:59.400619 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.400357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:36141 (local address 127.0.0.1:56784)
0808 02:22:59.400497 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.400551 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.253:36141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 02:22:59.400781 25192 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.253:36141: Network error: Client connection negotiation failed: client connection to 127.24.79.253:36141: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.253.keytab.
Entry for principal kudu/127.24.79.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.253.keytab.
I20260808 02:22:59.406644 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.253:36141
--webserver_interface=127.24.79.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.253.keytab
--principal=kudu/127.24.79.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:44173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:37055,127.24.79.253:36141,127.24.79.252:37011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:22:59.450091 25192 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 } attempt: 1
I20260808 02:22:59.451159 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.450763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:36141 (local address 127.0.0.1:56792)
0808 02:22:59.450993 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.451066 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.253:36141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 02:22:59.451983 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.451737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:36141 (local address 127.0.0.1:56800)
0808 02:22:59.451861 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.451914 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.253:36141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 02:22:59.452173 25192 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.253:36141: Network error: Client connection negotiation failed: client connection to 127.24.79.253:36141: connect: Connection refused (error 111)
I20260808 02:22:59.520395 25192 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 } attempt: 2
I20260808 02:22:59.521490 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.521077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:36141 (local address 127.0.0.1:56810)
0808 02:22:59.521328 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.521401 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.253:36141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260808 02:22:59.522348 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.522084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:36141 (local address 127.0.0.1:56826)
0808 02:22:59.522207 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.522260 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.253:36141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 02:22:59.522490 25192 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.253:36141: Network error: Client connection negotiation failed: client connection to 127.24.79.253:36141: connect: Connection refused (error 111)
W20260808 02:22:59.532073 25204 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 02:22:59.532447 25204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:59.532495 25204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:59.532563 25204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:59.537031 25204 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:59.537127 25204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:59.537155 25204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:59.537194 25204 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:59.537216 25204 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:59.543344 25204 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:44173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:37055,127.24.79.253:36141,127.24.79.252:37011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.253.keytab
--principal=kudu/127.24.79.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.253:36141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.79.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:59.544669 25204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:59.545882 25204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:22:59.556332 25220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:59.556475 25217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:59.556393 25204 server_base.cc:1034] running on GCE node
W20260808 02:22:59.556571 25218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:59.557307 25204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:59.558704 25204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:59.559921 25204 hybrid_clock.cc:648] HybridClock initialized: now 1786155779559907 us; error 67 us; skew 500 ppm
Aug 08 02:22:59 dist-test-slave-1vmg krb5kdc[25054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155779, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:59.563459 25204 init.cc:377] Logged in from keytab as kudu/127.24.79.253@KRBTEST.COM (short username kudu)
I20260808 02:22:59.564657 25204 webserver.cc:460] Webserver started at http://127.24.79.253:45567/ using document root <none> and password file <none>
I20260808 02:22:59.565286 25204 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:59.565357 25204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:59.565562 25204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:59.568030 25204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-1/data/instance:
uuid: "7210c7a0ccaa42d5944af6fbc9d172c8"
format_stamp: "Formatted at 2026-08-08 02:22:59 on dist-test-slave-1vmg"
server_key: "86cf8051e5772cf0714b6f2ef71647aa"
server_key_iv: "697401f4b0479a0231cdac147542206b"
server_key_version: "encryptionkey@0"
I20260808 02:22:59.568718 25204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-1/wal/instance:
uuid: "7210c7a0ccaa42d5944af6fbc9d172c8"
format_stamp: "Formatted at 2026-08-08 02:22:59 on dist-test-slave-1vmg"
server_key: "86cf8051e5772cf0714b6f2ef71647aa"
server_key_iv: "697401f4b0479a0231cdac147542206b"
server_key_version: "encryptionkey@0"
I20260808 02:22:59.573215 25204 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:22:59.576288 25230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:59.577824 25204 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:22:59.577973 25204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-1/wal
uuid: "7210c7a0ccaa42d5944af6fbc9d172c8"
format_stamp: "Formatted at 2026-08-08 02:22:59 on dist-test-slave-1vmg"
server_key: "86cf8051e5772cf0714b6f2ef71647aa"
server_key_iv: "697401f4b0479a0231cdac147542206b"
server_key_version: "encryptionkey@0"
I20260808 02:22:59.578120 25204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:59.597167 25204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:59.601094 25204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:59.601357 25204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:59.609187 25204 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.253:36141
I20260808 02:22:59.609479 25318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.253:36141 every 8 connection(s)
I20260808 02:22:59.610296 25204 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-1/data/info.pb
I20260808 02:22:59.613482 25319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:59.613675 25192 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 } attempt: 3
I20260808 02:22:59.615520 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 25204
I20260808 02:22:59.615660 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-1/wal/instance
I20260808 02:22:59.615988 24895 external_mini_cluster.cc:1428] Setting key ace5aa7bcf5d06da5b614504dd3c6d80
I20260808 02:22:59.619217 25319 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37055 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:22:59 dist-test-slave-1vmg krb5kdc[25054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155779, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for kudu/127.24.79.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.79.252@KRBTEST.COM; defaulting to no policy
I20260808 02:22:59.628842 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.614208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:36141 (local address 127.0.0.1:56830)
0808 02:22:59.614362 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.614379 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:59.614487 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:59.617400 (+ 2913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:59.617404 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:59.617626 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:59.617793 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:59.617802 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.619777 (+ 1975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.619782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:59.620595 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:59.620610 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.621757 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.621762 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:59.621901 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:59.622487 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:22:59.622503 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:59.624035 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:59.626541 (+ 2506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:59.626545 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:59.626547 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:59.626820 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:59.627347 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:59.627350 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:59.627352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:59.627461 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:59.628334 (+ 873us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:59.628337 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:59.628447 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 02:22:59.628546 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 02:22:59.628700 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 02:22:59.630045 25321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.615125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56830 (local address 127.24.79.253:36141)
0808 02:22:59.615578 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:59.615599 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:59.615652 (+ 53us) server_negotiation.cc:407] Connection header received
0808 02:22:59.616671 (+ 1019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:59.616695 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:59.617076 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:59.617432 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:59.617931 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.619536 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.620773 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.621591 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.622257 (+ 666us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:59.624244 (+ 1987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:59.624251 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:59.624267 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:59.624307 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:59.626392 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:59.627000 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:59.627007 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:59.627015 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:59.627176 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:59.627594 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:59.627598 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:59.627599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:59.628177 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:59.628317 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:59.628725 (+ 408us) server_negotiation.cc:299] Negotiation successful
0808 02:22:59.629037 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":117,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.252@KRBTEST.COM" created.
Aug 08 02:22:59 dist-test-slave-1vmg krb5kdc[25054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155779, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.253@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:22:59.632835 25192 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:22:59.634269 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.633937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:37011 (local address 127.0.0.1:50882)
0808 02:22:59.634100 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.634162 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:37011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 02:22:59.635053 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.634760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:37011 (local address 127.0.0.1:50898)
0808 02:22:59.634903 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.634981 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:37011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260808 02:22:59.635237 25192 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.252:37011: Network error: Client connection negotiation failed: client connection to 127.24.79.252:37011: connect: Connection refused (error 111)
I20260808 02:22:59.635679 25325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.621409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:37055 (local address 127.0.0.1:58452)
0808 02:22:59.621905 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.621958 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:59.622143 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:59.622574 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:59.622580 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:59.623022 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:59.623174 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:59.623181 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.624251 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.624254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:59.625131 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:59.625144 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.625815 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.625819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:59.625941 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:59.626489 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:22:59.626506 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:59.631278 (+ 4772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:59.633478 (+ 2200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:59.633492 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:59.633499 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:59.633762 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:59.634076 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:59.634079 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:59.634081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:59.634184 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:59.634961 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:59.634969 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:59.635130 (+ 161us) client_negotiation.cc:769] Sending connection context
0808 02:22:59.635321 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 02:22:59.635517 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":199,"threads_started":1}
I20260808 02:22:59.635720 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.621422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58452 (local address 127.24.79.254:37055)
0808 02:22:59.621554 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:59.621559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:59.622037 (+ 478us) server_negotiation.cc:407] Connection header received
0808 02:22:59.622259 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:59.622263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:59.622315 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:59.622370 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:59.623331 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.624123 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.625288 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.625673 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.625798 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:59.631493 (+ 5695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:59.631498 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:59.631501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:59.631534 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:59.633355 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:59.633872 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:59.633876 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:59.633878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:59.633964 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:59.634288 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:59.634292 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:59.634294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:59.634554 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:59.634662 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:59.635441 (+ 779us) server_negotiation.cc:299] Negotiation successful
0808 02:22:59.635580 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 02:22:59.637127 25319 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:22:59 dist-test-slave-1vmg krb5kdc[25054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155779, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.253@KRBTEST.COM for kudu/127.24.79.253@KRBTEST.COM
I20260808 02:22:59.647418 25325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.637683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:36141 (local address 127.0.0.1:56844)
0808 02:22:59.637797 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.637809 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:59.637881 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:59.638111 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:59.638114 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:59.638317 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:59.638419 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:59.638424 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.639197 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.639201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:59.639743 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:59.639751 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.640195 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.640199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:59.640305 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:59.640815 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:22:59.640828 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:59.643835 (+ 3007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:59.645758 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:59.645761 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:59.645763 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:59.645965 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:59.646236 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:59.646239 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:59.646241 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:59.646323 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:59.646715 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:59.646718 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:59.646838 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 02:22:59.646978 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 02:22:59.647158 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 02:22:59.647418 25321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.637841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56844 (local address 127.24.79.253:36141)
0808 02:22:59.637949 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:59.637953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:59.637964 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:22:59.637992 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:59.637995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:59.638028 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:59.638088 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:59.638526 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.639105 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.639865 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.640095 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.640185 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:59.644034 (+ 3849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:59.644037 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:59.644039 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:59.644068 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:59.645622 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:59.646044 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:59.646047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:59.646048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:59.646128 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:59.646406 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:59.646410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:59.646412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:59.646613 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:59.646701 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:59.647033 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 02:22:59.647264 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":89,"server-negotiator.queue_time_us":43}
I20260808 02:22:59.648464 25319 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.252.keytab.
Entry for principal kudu/127.24.79.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.252.keytab.
I20260808 02:22:59.649533 25325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.649049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:37011 (local address 127.0.0.1:50904)
0808 02:22:59.649322 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.649442 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:37011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260808 02:22:59.650401 25237 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.79.252:37011: connect: Connection refused (error 111)
I20260808 02:22:59.650686 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.252:37011
--webserver_interface=127.24.79.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.252.keytab
--principal=kudu/127.24.79.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:44173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:37055,127.24.79.253:36141,127.24.79.252:37011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:22:59.651661 25325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.651392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:37011 (local address 127.0.0.1:50916)
0808 02:22:59.651532 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.651586 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:37011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 02:22:59.651822 25319 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.252:37011: Network error: Client connection negotiation failed: client connection to 127.24.79.252:37011: connect: Connection refused (error 111)
I20260808 02:22:59.666450 25192 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } attempt: 1
I20260808 02:22:59.667567 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.667175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:37011 (local address 127.0.0.1:50926)
0808 02:22:59.667351 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.667429 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:37011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 02:22:59.668473 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.668212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:37011 (local address 127.0.0.1:50932)
0808 02:22:59.668342 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.668399 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:37011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 02:22:59.668639 25192 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.252:37011: Network error: Client connection negotiation failed: client connection to 127.24.79.252:37011: connect: Connection refused (error 111)
I20260808 02:22:59.701146 25319 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } attempt: 1
I20260808 02:22:59.702342 25325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.701952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:37011 (local address 127.0.0.1:50942)
0808 02:22:59.702172 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.702247 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:37011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260808 02:22:59.703325 25325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.703058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:37011 (local address 127.0.0.1:50958)
0808 02:22:59.703188 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.703244 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:37011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 02:22:59.703464 25319 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.252:37011: Network error: Client connection negotiation failed: client connection to 127.24.79.252:37011: connect: Connection refused (error 111)
I20260808 02:22:59.743832 25192 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } attempt: 2
I20260808 02:22:59.744781 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.744461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:37011 (local address 127.0.0.1:50966)
0808 02:22:59.744624 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.744693 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:37011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 02:22:59.745581 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.745328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:37011 (local address 127.0.0.1:50982)
0808 02:22:59.745456 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.745506 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:37011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 02:22:59.745712 25192 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.252:37011: Network error: Client connection negotiation failed: client connection to 127.24.79.252:37011: connect: Connection refused (error 111)
I20260808 02:22:59.771674 25319 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } attempt: 2
I20260808 02:22:59.772719 25325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.772341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:37011 (local address 127.0.0.1:50986)
0808 02:22:59.772541 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.772621 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:37011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260808 02:22:59.773670 25325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.773392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:37011 (local address 127.0.0.1:50996)
0808 02:22:59.773535 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.773588 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:37011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260808 02:22:59.773813 25319 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.252:37011: Network error: Client connection negotiation failed: client connection to 127.24.79.252:37011: connect: Connection refused (error 111)
W20260808 02:22:59.782975 25328 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 02:22:59.783336 25328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:22:59.783378 25328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:22:59.783445 25328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:22:59.787747 25328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:22:59.787845 25328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:22:59.787873 25328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:22:59.787896 25328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:22:59.787928 25328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:22:59.793884 25328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:44173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:37055,127.24.79.253:36141,127.24.79.252:37011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.252.keytab
--principal=kudu/127.24.79.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.252:37011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.79.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:22:59.795267 25328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:22:59.796437 25328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:22:59.806691 25328 server_base.cc:1034] running on GCE node
W20260808 02:22:59.806622 25338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:59.806622 25335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:22:59.806622 25340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:22:59.807767 25328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:22:59.809203 25328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:22:59.810447 25328 hybrid_clock.cc:648] HybridClock initialized: now 1786155779810388 us; error 74 us; skew 500 ppm
Aug 08 02:22:59 dist-test-slave-1vmg krb5kdc[25054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155779, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:22:59.814033 25328 init.cc:377] Logged in from keytab as kudu/127.24.79.252@KRBTEST.COM (short username kudu)
I20260808 02:22:59.815291 25328 webserver.cc:460] Webserver started at http://127.24.79.252:38189/ using document root <none> and password file <none>
I20260808 02:22:59.815984 25328 fs_manager.cc:359] Metadata directory not provided
I20260808 02:22:59.816079 25328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:22:59.816465 25328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:22:59.818902 25328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-2/data/instance:
uuid: "5823f7194dca4193beb06ed2cc7d834c"
format_stamp: "Formatted at 2026-08-08 02:22:59 on dist-test-slave-1vmg"
server_key: "e824870336eda4e10e0f5e05c4d510e3"
server_key_iv: "015cf6c89660b792ac1e055f3e147d52"
server_key_version: "encryptionkey@0"
I20260808 02:22:59.819681 25328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-2/wal/instance:
uuid: "5823f7194dca4193beb06ed2cc7d834c"
format_stamp: "Formatted at 2026-08-08 02:22:59 on dist-test-slave-1vmg"
server_key: "e824870336eda4e10e0f5e05c4d510e3"
server_key_iv: "015cf6c89660b792ac1e055f3e147d52"
server_key_version: "encryptionkey@0"
I20260808 02:22:59.824362 25328 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:22:59.827598 25347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:22:59.829243 25328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:22:59.829492 25328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-2/wal
uuid: "5823f7194dca4193beb06ed2cc7d834c"
format_stamp: "Formatted at 2026-08-08 02:22:59 on dist-test-slave-1vmg"
server_key: "e824870336eda4e10e0f5e05c4d510e3"
server_key_iv: "015cf6c89660b792ac1e055f3e147d52"
server_key_version: "encryptionkey@0"
I20260808 02:22:59.829676 25328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:22:59.841636 25328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:22:59.844897 25192 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } attempt: 3
I20260808 02:22:59.845491 25328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:22:59.845715 25328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:22:59.845871 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.845490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:37011 (local address 127.0.0.1:51010)
0808 02:22:59.845699 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.845770 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:37011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 02:22:59.846813 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.846526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:37011 (local address 127.0.0.1:51012)
0808 02:22:59.846663 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.846717 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:37011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260808 02:22:59.847005 25192 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.252:37011: Network error: Client connection negotiation failed: client connection to 127.24.79.252:37011: connect: Connection refused (error 111)
I20260808 02:22:59.853797 25328 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.252:37011
I20260808 02:22:59.854302 25432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.252:37011 every 8 connection(s)
I20260808 02:22:59.855254 25328 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-2/data/info.pb
I20260808 02:22:59.858644 25433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:22:59.860213 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 25328
I20260808 02:22:59.860365 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155464663577-24895-0/minicluster-data/master-2/wal/instance
I20260808 02:22:59.860692 24895 external_mini_cluster.cc:1428] Setting key c20ead291cc78ecb2425742feeff3ac9
I20260808 02:22:59.863783 25433 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37055 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:22:59.864995 25319 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } attempt: 3
Aug 08 02:22:59 dist-test-slave-1vmg krb5kdc[25054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.252@KRBTEST.COM
Aug 08 02:22:59 dist-test-slave-1vmg krb5kdc[25054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155779, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.253@KRBTEST.COM for kudu/127.24.79.252@KRBTEST.COM
Aug 08 02:22:59 dist-test-slave-1vmg krb5kdc[25054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155779, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.252@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:22:59.877117 25325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.865535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:37011 (local address 127.0.0.1:51032)
0808 02:22:59.865688 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.865703 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:59.865781 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:59.867265 (+ 1484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:59.867268 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:59.867434 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:59.867547 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:59.867555 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.868696 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.868701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:59.869472 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:59.869487 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.870122 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.870127 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:59.870223 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:59.870830 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:22:59.870846 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:59.872588 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:59.875420 (+ 2832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:59.875423 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:59.875425 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:59.875683 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:59.875968 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:59.875971 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:59.875973 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:59.876068 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:59.876558 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:59.876561 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:59.876684 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 02:22:59.876769 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 02:22:59.876932 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 02:22:59.877923 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.865923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58462 (local address 127.24.79.254:37055)
0808 02:22:59.866074 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:59.866079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:59.866451 (+ 372us) server_negotiation.cc:407] Connection header received
0808 02:22:59.866606 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:59.866610 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:59.866666 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:59.866763 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:59.867667 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.868161 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.869401 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.869750 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.869831 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:59.873517 (+ 3686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:59.873521 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:59.873524 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:59.873559 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:59.875280 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:59.876242 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:59.876247 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:59.876248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:59.876349 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:59.876664 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:59.876668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:59.876669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:59.876910 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:59.877018 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:59.877643 (+ 625us) server_negotiation.cc:299] Negotiation successful
0808 02:22:59.877768 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 02:22:59.878598 25438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.865840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:37055 (local address 127.0.0.1:58462)
0808 02:22:59.866308 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.866362 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:59.866501 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:59.866826 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:59.866832 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:59.867382 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:59.867534 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:59.867542 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.868287 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.868291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:59.869250 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:59.869262 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.870046 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.870051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:59.870195 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:59.870797 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:22:59.870820 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:59.873319 (+ 2499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:59.875810 (+ 2491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:59.875818 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:59.875825 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:59.876087 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:59.876467 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:59.876470 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:59.876472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:59.876559 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:59.877268 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:59.877276 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:59.877408 (+ 132us) client_negotiation.cc:769] Sending connection context
0808 02:22:59.877547 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 02:22:59.877764 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":5,"thread_start_us":170,"threads_started":1}
I20260808 02:22:59.878631 25440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.866608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51032 (local address 127.24.79.252:37011)
0808 02:22:59.866964 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:59.866972 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:59.866984 (+ 12us) server_negotiation.cc:407] Connection header received
0808 02:22:59.867017 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:59.867019 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:59.867055 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:59.867146 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:59.867702 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.868530 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.869622 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.869950 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.870126 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:59.872763 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:59.872767 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:59.872770 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:59.872804 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:59.875285 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:59.875781 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:59.875783 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:59.875785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:59.875864 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:59.876161 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:59.876164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:59.876166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:59.876444 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:59.877573 (+ 1129us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:59.877648 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 02:22:59.877835 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":262,"thread_start_us":113,"threads_started":1}
I20260808 02:22:59.878612 25437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.862404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51022 (local address 127.24.79.252:37011)
0808 02:22:59.862973 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:59.863002 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:59.863047 (+ 45us) server_negotiation.cc:407] Connection header received
0808 02:22:59.863757 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:59.863776 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:59.864171 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:59.864523 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:59.865495 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.867383 (+ 1888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.868687 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.869211 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.869369 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:59.872538 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:59.872547 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:59.872567 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:59.872598 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:59.874617 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:59.875170 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:59.875178 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:59.875187 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:59.875315 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:59.875650 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:59.875654 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:59.875656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:59.876313 (+ 657us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:59.876515 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:59.876878 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 02:22:59.877470 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":164,"threads_started":1}
I20260808 02:22:59.880750 25433 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:22:59.883924 25319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7210c7a0ccaa42d5944af6fbc9d172c8: Bootstrap starting.
Aug 08 02:22:59 dist-test-slave-1vmg krb5kdc[25054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155779, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.252@KRBTEST.COM for kudu/127.24.79.253@KRBTEST.COM
I20260808 02:22:59.887164 25319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7210c7a0ccaa42d5944af6fbc9d172c8: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:59.888185 25319 log.cc:824] T 00000000000000000000000000000000 P 7210c7a0ccaa42d5944af6fbc9d172c8: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:22:59 dist-test-slave-1vmg krb5kdc[25054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:22:59.890581 25321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.881452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56860 (local address 127.24.79.253:36141)
0808 02:22:59.881579 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:59.881583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:59.881598 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:22:59.881640 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:59.881642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:59.881691 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:59.881774 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:59.882241 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.883018 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.884300 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.884528 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.884619 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:59.886757 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:59.886760 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:59.886763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:59.886794 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:59.888583 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:59.889119 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:59.889123 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:59.889125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:59.889243 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:59.889592 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:59.889596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:59.889598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:59.889872 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:59.889983 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:59.890303 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 02:22:59.890434 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 02:22:59.890583 25438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.881331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:36141 (local address 127.0.0.1:56860)
0808 02:22:59.881440 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.881455 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:59.881576 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:59.881795 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:59.881798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:59.881988 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:59.882101 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:59.882107 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.883170 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.883174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:59.884183 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:59.884195 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.884633 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.884636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:59.884744 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:59.885235 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:22:59.885251 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:59.886608 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:59.888724 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:59.888728 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:59.888730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:59.888989 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:59.889380 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:59.889383 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:59.889384 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:59.889488 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:59.889986 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:59.889990 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:59.890111 (+ 121us) client_negotiation.cc:769] Sending connection context
0808 02:22:59.890233 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 02:22:59.890393 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 02:22:59.891088 25319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7210c7a0ccaa42d5944af6fbc9d172c8: No bootstrap required, opened a new log
I20260808 02:22:59.891453 25433 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:22:59.893262 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.883541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58464 (local address 127.24.79.254:37055)
0808 02:22:59.883666 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:59.883670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:59.883685 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:22:59.883726 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:59.883729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:59.883773 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:59.883855 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:59.884682 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.885385 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.886512 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.886848 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.886969 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:59.889553 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:59.889557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:59.889560 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:59.889591 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:59.891486 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:59.891933 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:59.891936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:59.891938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:59.892044 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:59.892369 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:59.892373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:59.892375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:59.892589 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:59.892838 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:59.892979 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 02:22:59.893100 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 02:22:59.895126 25319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7210c7a0ccaa42d5944af6fbc9d172c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e73934c54a84eafb35b93675ab9a92c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37055 } } peers { permanent_uuid: "7210c7a0ccaa42d5944af6fbc9d172c8" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 } } peers { permanent_uuid: "5823f7194dca4193beb06ed2cc7d834c" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } }
I20260808 02:22:59.895367 25319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7210c7a0ccaa42d5944af6fbc9d172c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7210c7a0ccaa42d5944af6fbc9d172c8, State: Initialized, Role: FOLLOWER
I20260808 02:22:59.895741 25319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7210c7a0ccaa42d5944af6fbc9d172c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e73934c54a84eafb35b93675ab9a92c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37055 } } peers { permanent_uuid: "7210c7a0ccaa42d5944af6fbc9d172c8" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 } } peers { permanent_uuid: "5823f7194dca4193beb06ed2cc7d834c" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } }
I20260808 02:22:59.896279 25443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7210c7a0ccaa42d5944af6fbc9d172c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e73934c54a84eafb35b93675ab9a92c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37055 } } peers { permanent_uuid: "7210c7a0ccaa42d5944af6fbc9d172c8" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 } } peers { permanent_uuid: "5823f7194dca4193beb06ed2cc7d834c" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } } }
I20260808 02:22:59.896436 25443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7210c7a0ccaa42d5944af6fbc9d172c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:22:59.897292 25319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7210c7a0ccaa42d5944af6fbc9d172c8 [sys.catalog]: configured and running, proceeding with master startup.
Aug 08 02:22:59 dist-test-slave-1vmg krb5kdc[25054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.253@KRBTEST.COM
Aug 08 02:22:59 dist-test-slave-1vmg krb5kdc[25054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155779, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.252@KRBTEST.COM for kudu/127.24.79.252@KRBTEST.COM
W20260808 02:22:59.905074 25459 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7210c7a0ccaa42d5944af6fbc9d172c8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:22:59.905181 25459 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:22:59.905450 25321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.894502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56876 (local address 127.24.79.253:36141)
0808 02:22:59.894615 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:59.894619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:59.894632 (+ 13us) server_negotiation.cc:407] Connection header received
0808 02:22:59.894751 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:59.894755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:59.894804 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:59.894870 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:59.895573 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.896195 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.897291 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.897579 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.897676 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:59.900965 (+ 3289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:59.900969 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:59.900972 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:59.901003 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:59.902846 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:59.903455 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:59.903460 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:59.903462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:59.903634 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:59.904183 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:59.904187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:59.904189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:59.904507 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:59.904683 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:59.905119 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 02:22:59.905282 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 02:22:59.902233 25204 master_runner.cc:186] Error getting master registration for 127.24.79.254:37055: OK, instance_id { permanent_uuid: "3e73934c54a84eafb35b93675ab9a92c" instance_seqno: 1786155779366036 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 02:22:59.906107 25204 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:22:59.907995 25438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.892042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:37011 (local address 127.0.0.1:51040)
0808 02:22:59.892250 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 02:22:59.892264 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:22:59.892340 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:22:59.899364 (+ 7024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:22:59.899367 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:22:59.899535 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:22:59.899649 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:59.899658 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.900696 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.900699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:59.901409 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:22:59.901421 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.902089 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.902093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:22:59.902182 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:59.902631 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:22:59.902647 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:22:59.904300 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:22:59.906284 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:59.906287 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:59.906289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:59.906525 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:59.906871 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:22:59.906875 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:22:59.906877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:22:59.906998 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:22:59.907431 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:22:59.907435 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:22:59.907532 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 02:22:59.907643 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 02:22:59.907798 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135}
I20260808 02:22:59.908092 25437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:22:59.892117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51040 (local address 127.24.79.252:37011)
0808 02:22:59.899079 (+ 6962us) server_negotiation.cc:206] Beginning negotiation
0808 02:22:59.899085 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:22:59.899103 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:22:59.899154 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:22:59.899157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:22:59.899213 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:22:59.899308 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:22:59.899835 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.900561 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.901530 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:22:59.901866 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:22:59.901943 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:22:59.904453 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:22:59.904457 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:22:59.904459 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:22:59.904484 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:22:59.906155 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:59.906645 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:59.906649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:59.906651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:59.906769 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:22:59.907086 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:22:59.907089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:22:59.907091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:22:59.907333 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:22:59.907427 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 02:22:59.907744 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 02:22:59.907883 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6841}
I20260808 02:22:59.910451 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:22:59.912089 25433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5823f7194dca4193beb06ed2cc7d834c: Bootstrap starting.
I20260808 02:22:59.911931 25145 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3e73934c54a84eafb35b93675ab9a92c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58464
I20260808 02:22:59.915356 25433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5823f7194dca4193beb06ed2cc7d834c: Neither blocks nor log segments found. Creating new log.
I20260808 02:22:59.916455 25433 log.cc:824] T 00000000000000000000000000000000 P 5823f7194dca4193beb06ed2cc7d834c: Log is configured to *not* fsync() on all Append() calls
I20260808 02:22:59.919350 25433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5823f7194dca4193beb06ed2cc7d834c: No bootstrap required, opened a new log
I20260808 02:22:59.923910 25433 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5823f7194dca4193beb06ed2cc7d834c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e73934c54a84eafb35b93675ab9a92c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37055 } } peers { permanent_uuid: "7210c7a0ccaa42d5944af6fbc9d172c8" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 } } peers { permanent_uuid: "5823f7194dca4193beb06ed2cc7d834c" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } }
I20260808 02:22:59.924263 25433 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5823f7194dca4193beb06ed2cc7d834c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5823f7194dca4193beb06ed2cc7d834c, State: Initialized, Role: FOLLOWER
I20260808 02:22:59.924667 25433 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5823f7194dca4193beb06ed2cc7d834c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e73934c54a84eafb35b93675ab9a92c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37055 } } peers { permanent_uuid: "7210c7a0ccaa42d5944af6fbc9d172c8" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 } } peers { permanent_uuid: "5823f7194dca4193beb06ed2cc7d834c" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } }
I20260808 02:22:59.925312 25462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5823f7194dca4193beb06ed2cc7d834c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e73934c54a84eafb35b93675ab9a92c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37055 } } peers { permanent_uuid: "7210c7a0ccaa42d5944af6fbc9d172c8" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 } } peers { permanent_uuid: "5823f7194dca4193beb06ed2cc7d834c" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } } }
I20260808 02:22:59.925486 25462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5823f7194dca4193beb06ed2cc7d834c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:22:59.926389 25433 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5823f7194dca4193beb06ed2cc7d834c [sys.catalog]: configured and running, proceeding with master startup.
W20260808 02:22:59.933274 25480 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5823f7194dca4193beb06ed2cc7d834c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:22:59.933466 25480 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:22:59.931496 25328 master_runner.cc:186] Error getting master registration for 127.24.79.254:37055: OK, instance_id { permanent_uuid: "3e73934c54a84eafb35b93675ab9a92c" instance_seqno: 1786155779366036 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 02:22:59.935050 25328 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:23:00.011191 25192 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } attempt: 4
Aug 08 02:23:00 dist-test-slave-1vmg krb5kdc[25054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155779, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for kudu/127.24.79.252@KRBTEST.COM
I20260808 02:23:00.023309 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:00.011952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:37011 (local address 127.0.0.1:51046)
0808 02:23:00.012366 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:00.012384 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:23:00.012497 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:23:00.012805 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:23:00.012808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:23:00.013053 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:23:00.013187 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:23:00.013195 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:23:00.014335 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:23:00.014339 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:23:00.015259 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:23:00.015276 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:23:00.016166 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:23:00.016170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:23:00.016274 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:00.017051 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:23:00.017069 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:23:00.018967 (+ 1898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:23:00.021352 (+ 2385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:23:00.021356 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:23:00.021358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:23:00.021678 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:23:00.022023 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:23:00.022026 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:23:00.022028 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:23:00.022122 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:23:00.022643 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:23:00.022646 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:23:00.022784 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 02:23:00.022904 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 02:23:00.023110 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307}
I20260808 02:23:00.024171 25437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:00.012370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51046 (local address 127.24.79.252:37011)
0808 02:23:00.012521 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:00.012526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:00.012545 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:23:00.012610 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:00.012614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:00.012665 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:00.012773 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:00.013347 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:00.014199 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:00.015442 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:00.015859 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:00.015968 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:00.019197 (+ 3229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:23:00.019201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:23:00.019204 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:23:00.019242 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:23:00.021159 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:00.021809 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:00.021814 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:00.021816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:00.021920 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:00.022227 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:00.022230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:00.022232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:00.022495 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:23:00.022592 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:00.023848 (+ 1256us) server_negotiation.cc:299] Negotiation successful
0808 02:23:00.024005 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 02:23:00.028697 25192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c: Bootstrap starting.
I20260808 02:23:00.032099 25192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c: Neither blocks nor log segments found. Creating new log.
I20260808 02:23:00.033473 25192 log.cc:824] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c: Log is configured to *not* fsync() on all Append() calls
I20260808 02:23:00.036917 25192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c: No bootstrap required, opened a new log
I20260808 02:23:00.040693 25192 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e73934c54a84eafb35b93675ab9a92c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37055 } } peers { permanent_uuid: "7210c7a0ccaa42d5944af6fbc9d172c8" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 } } peers { permanent_uuid: "5823f7194dca4193beb06ed2cc7d834c" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } }
I20260808 02:23:00.041018 25192 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e73934c54a84eafb35b93675ab9a92c, State: Initialized, Role: FOLLOWER
I20260808 02:23:00.041422 25192 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e73934c54a84eafb35b93675ab9a92c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37055 } } peers { permanent_uuid: "7210c7a0ccaa42d5944af6fbc9d172c8" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 } } peers { permanent_uuid: "5823f7194dca4193beb06ed2cc7d834c" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } }
I20260808 02:23:00.042157 25485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e73934c54a84eafb35b93675ab9a92c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37055 } } peers { permanent_uuid: "7210c7a0ccaa42d5944af6fbc9d172c8" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 } } peers { permanent_uuid: "5823f7194dca4193beb06ed2cc7d834c" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } } }
I20260808 02:23:00.042490 25485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:23:00.043177 25192 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:23:00.043237 25145 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 02:23:00.043363 25145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:23:00.044785 25145 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e73934c54a84eafb35b93675ab9a92c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37055 } } peers { permanent_uuid: "7210c7a0ccaa42d5944af6fbc9d172c8" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 } } peers { permanent_uuid: "5823f7194dca4193beb06ed2cc7d834c" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } }
I20260808 02:23:00.045856 25145 leader_election.cc:288] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c [CANDIDATE]: Term 1 election: Requested vote from peers 7210c7a0ccaa42d5944af6fbc9d172c8 (127.24.79.253:36141), 5823f7194dca4193beb06ed2cc7d834c (127.24.79.252:37011)
I20260808 02:23:00.048138 25278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e73934c54a84eafb35b93675ab9a92c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7210c7a0ccaa42d5944af6fbc9d172c8"
I20260808 02:23:00.048436 25278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7210c7a0ccaa42d5944af6fbc9d172c8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:23:00.049932 25278 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7210c7a0ccaa42d5944af6fbc9d172c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e73934c54a84eafb35b93675ab9a92c in term 1.
W20260808 02:23:00.050182 25499 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:23:00.050263 25499 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:23:00.050493 25392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e73934c54a84eafb35b93675ab9a92c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5823f7194dca4193beb06ed2cc7d834c"
I20260808 02:23:00.050768 25392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5823f7194dca4193beb06ed2cc7d834c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:23:00.052102 25392 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5823f7194dca4193beb06ed2cc7d834c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e73934c54a84eafb35b93675ab9a92c in term 1.
I20260808 02:23:00.054761 25104 leader_election.cc:302] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e73934c54a84eafb35b93675ab9a92c, 7210c7a0ccaa42d5944af6fbc9d172c8; no voters:
I20260808 02:23:00.062443 25081 master_runner.cc:263] No leader master found from master 3e73934c54a84eafb35b93675ab9a92c
I20260808 02:23:00.062556 25081 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:23:00.062816 25485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:23:00.063232 25485 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c [term 1 LEADER]: Becoming Leader. State: Replica: 3e73934c54a84eafb35b93675ab9a92c, State: Running, Role: LEADER
I20260808 02:23:00.063728 25485 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e73934c54a84eafb35b93675ab9a92c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37055 } } peers { permanent_uuid: "7210c7a0ccaa42d5944af6fbc9d172c8" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 } } peers { permanent_uuid: "5823f7194dca4193beb06ed2cc7d834c" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } }
I20260808 02:23:00.066267 25485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e73934c54a84eafb35b93675ab9a92c. Latest consensus state: current_term: 1 leader_uuid: "3e73934c54a84eafb35b93675ab9a92c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e73934c54a84eafb35b93675ab9a92c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37055 } } peers { permanent_uuid: "7210c7a0ccaa42d5944af6fbc9d172c8" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 } } peers { permanent_uuid: "5823f7194dca4193beb06ed2cc7d834c" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } } }
I20260808 02:23:00.066516 25485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c [sys.catalog]: This master's current role is: LEADER
I20260808 02:23:00.069965 25511 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:23:00.073638 25511 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:23:00.089641 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:00.069144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58472 (local address 127.24.79.254:37055)
0808 02:23:00.069275 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:00.069280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:00.079470 (+ 10190us) server_negotiation.cc:407] Connection header received
0808 02:23:00.079702 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:00.079706 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:00.079783 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:00.079843 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:00.080971 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:00.081674 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:00.082806 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:00.083201 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:00.083288 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:00.085379 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:23:00.085383 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:23:00.085386 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:23:00.085423 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:23:00.087454 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:00.088032 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:00.088037 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:00.088039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:00.088172 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:00.088541 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:00.088545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:00.088547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:00.088844 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:23:00.088996 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:00.089296 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 02:23:00.089449 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 02:23:00.095729 25437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:00.068254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51048 (local address 127.24.79.252:37011)
0808 02:23:00.068385 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:00.068390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:00.074290 (+ 5900us) server_negotiation.cc:407] Connection header received
0808 02:23:00.075959 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:00.075963 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:00.076021 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:00.076077 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:00.081677 (+ 5600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:00.082467 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:00.084334 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:00.084699 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:00.084799 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:00.091757 (+ 6958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:23:00.091761 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:23:00.091764 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:23:00.091799 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:23:00.093518 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:00.094083 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:00.094087 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:00.094089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:00.094195 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:00.094522 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:00.094525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:00.094526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:00.094790 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:23:00.094977 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:00.095400 (+ 423us) server_negotiation.cc:299] Negotiation successful
0808 02:23:00.095544 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 02:23:00.097610 25360 tablet.cc:2369] T 00000000000000000000000000000000 P 5823f7194dca4193beb06ed2cc7d834c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:23:00.103787 25241 tablet.cc:2369] T 00000000000000000000000000000000 P 7210c7a0ccaa42d5944af6fbc9d172c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:23:00.104022 25392 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5823f7194dca4193beb06ed2cc7d834c [term 1 FOLLOWER]: Refusing update from remote peer 3e73934c54a84eafb35b93675ab9a92c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:23:00.105036 25508 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5823f7194dca4193beb06ed2cc7d834c" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:23:00.110257 25321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:00.067409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56890 (local address 127.24.79.253:36141)
0808 02:23:00.067547 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:00.067552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:00.072484 (+ 4932us) server_negotiation.cc:407] Connection header received
0808 02:23:00.074400 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:00.074404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:00.074470 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:00.074527 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:00.080256 (+ 5729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:00.081034 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:00.084998 (+ 3964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:00.085352 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:00.085434 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:00.103126 (+ 17692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:23:00.103132 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:23:00.103135 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:23:00.103192 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:23:00.107551 (+ 4359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:00.108225 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:00.108230 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:00.108231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:00.108351 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:00.108715 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:00.108719 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:00.108721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:00.108982 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:23:00.109118 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:00.109916 (+ 798us) server_negotiation.cc:299] Negotiation successful
0808 02:23:00.110074 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 02:23:00.112385 25111 tablet.cc:2369] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:23:00.113933 25462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5823f7194dca4193beb06ed2cc7d834c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e73934c54a84eafb35b93675ab9a92c. Latest consensus state: current_term: 1 leader_uuid: "3e73934c54a84eafb35b93675ab9a92c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e73934c54a84eafb35b93675ab9a92c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37055 } } peers { permanent_uuid: "7210c7a0ccaa42d5944af6fbc9d172c8" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 } } peers { permanent_uuid: "5823f7194dca4193beb06ed2cc7d834c" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } } }
I20260808 02:23:00.114117 25462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5823f7194dca4193beb06ed2cc7d834c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:23:00.115681 25278 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7210c7a0ccaa42d5944af6fbc9d172c8 [term 1 FOLLOWER]: Refusing update from remote peer 3e73934c54a84eafb35b93675ab9a92c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:23:00.116572 25508 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7210c7a0ccaa42d5944af6fbc9d172c8" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:23:00.121701 25485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3e73934c54a84eafb35b93675ab9a92c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e73934c54a84eafb35b93675ab9a92c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37055 } } peers { permanent_uuid: "7210c7a0ccaa42d5944af6fbc9d172c8" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 } } peers { permanent_uuid: "5823f7194dca4193beb06ed2cc7d834c" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } } }
I20260808 02:23:00.121891 25485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c [sys.catalog]: This master's current role is: LEADER
I20260808 02:23:00.122128 25517 mvcc.cc:205] Tried to move back new op lower bound from 7316094075293196288 to 7316094075146407936. Current Snapshot: MvccSnapshot[applied={T|T < 7316094075293196288}]
I20260808 02:23:00.122431 25462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5823f7194dca4193beb06ed2cc7d834c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3e73934c54a84eafb35b93675ab9a92c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e73934c54a84eafb35b93675ab9a92c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37055 } } peers { permanent_uuid: "7210c7a0ccaa42d5944af6fbc9d172c8" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 } } peers { permanent_uuid: "5823f7194dca4193beb06ed2cc7d834c" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } } }
I20260808 02:23:00.122571 25462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5823f7194dca4193beb06ed2cc7d834c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:23:00.126150 25443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7210c7a0ccaa42d5944af6fbc9d172c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e73934c54a84eafb35b93675ab9a92c. Latest consensus state: current_term: 1 leader_uuid: "3e73934c54a84eafb35b93675ab9a92c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e73934c54a84eafb35b93675ab9a92c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37055 } } peers { permanent_uuid: "7210c7a0ccaa42d5944af6fbc9d172c8" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 } } peers { permanent_uuid: "5823f7194dca4193beb06ed2cc7d834c" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } } }
I20260808 02:23:00.126361 25443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7210c7a0ccaa42d5944af6fbc9d172c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:23:00.127658 25511 catalog_manager.cc:1347] Generated new cluster ID: 6aee7998ccb44e7194bed103bf1d8eae
I20260808 02:23:00.127815 25511 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:23:00.130612 25508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3e73934c54a84eafb35b93675ab9a92c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e73934c54a84eafb35b93675ab9a92c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37055 } } peers { permanent_uuid: "7210c7a0ccaa42d5944af6fbc9d172c8" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 } } peers { permanent_uuid: "5823f7194dca4193beb06ed2cc7d834c" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } } }
I20260808 02:23:00.130825 25508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c [sys.catalog]: This master's current role is: LEADER
I20260808 02:23:00.132457 25528 mvcc.cc:205] Tried to move back new op lower bound from 7316094075293196288 to 7316094075146407936. Current Snapshot: MvccSnapshot[applied={T|T < 7316094075293196288}]
I20260808 02:23:00.132457 25443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7210c7a0ccaa42d5944af6fbc9d172c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3e73934c54a84eafb35b93675ab9a92c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e73934c54a84eafb35b93675ab9a92c" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 37055 } } peers { permanent_uuid: "7210c7a0ccaa42d5944af6fbc9d172c8" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 36141 } } peers { permanent_uuid: "5823f7194dca4193beb06ed2cc7d834c" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 37011 } } }
I20260808 02:23:00.132905 25443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7210c7a0ccaa42d5944af6fbc9d172c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:23:00.156620 25511 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:23:00.158120 25511 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:23:00.170717 25511 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e73934c54a84eafb35b93675ab9a92c: Generated new TSK 0
I20260808 02:23:00.172848 25511 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 02:23:00.211474 25198 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:00.209014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58488 (local address 127.24.79.254:37055)
0808 02:23:00.209158 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:00.209163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:00.209763 (+ 600us) server_negotiation.cc:407] Connection header received
0808 02:23:00.210303 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:00.210307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:00.210380 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:00.210485 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:00.211307 (+ 822us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58488: BlockingRecv error: recv got EOF from 127.0.0.1:58488 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 02:23:00.211577 25321 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:00.209062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56896 (local address 127.24.79.253:36141)
0808 02:23:00.209185 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:00.209189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:00.210173 (+ 984us) server_negotiation.cc:407] Connection header received
0808 02:23:00.210240 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:00.210244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:00.210305 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:00.210394 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:00.211420 (+ 1026us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56896: BlockingRecv error: recv got EOF from 127.0.0.1:56896 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 02:23:00.211577 25437 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:00.209996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51056 (local address 127.24.79.252:37011)
0808 02:23:00.210129 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:00.210133 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:00.210418 (+ 285us) server_negotiation.cc:407] Connection header received
0808 02:23:00.210603 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:00.210607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:00.210662 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:00.210718 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:00.211422 (+ 704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51056: BlockingRecv error: recv got EOF from 127.0.0.1:51056 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 02:23:00.215577 25321 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:00.214099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56902 (local address 127.24.79.253:36141)
0808 02:23:00.214224 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:00.214229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:00.214559 (+ 330us) server_negotiation.cc:407] Connection header received
0808 02:23:00.214739 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:00.214742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:00.214833 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:00.214947 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:00.215444 (+ 497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56902: BlockingRecv error: recv got EOF from 127.0.0.1:56902 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 02:23:00.220280 25321 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:00.219190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56918 (local address 127.24.79.253:36141)
0808 02:23:00.219369 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:00.219374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:00.219393 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:23:00.219455 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:00.219458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:00.219509 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:00.219610 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:00.220138 (+ 528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56918: BlockingRecv error: recv got EOF from 127.0.0.1:56918 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260808 02:23:00.225425 25321 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:00.224288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56920 (local address 127.24.79.253:36141)
0808 02:23:00.224424 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:00.224429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:00.224528 (+ 99us) server_negotiation.cc:407] Connection header received
0808 02:23:00.224658 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:00.224661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:00.224729 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:00.224831 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:00.225288 (+ 457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56920: BlockingRecv error: recv got EOF from 127.0.0.1:56920 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 02:23:00.232935 25321 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:00.231611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56930 (local address 127.24.79.253:36141)
0808 02:23:00.231829 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:00.231833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:00.231939 (+ 106us) server_negotiation.cc:407] Connection header received
0808 02:23:00.232109 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:00.232112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:00.232162 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:00.232271 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:00.232797 (+ 526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56930: BlockingRecv error: recv got EOF from 127.0.0.1:56930 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
W20260808 02:23:00.244504 25321 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:00.243269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56938 (local address 127.24.79.253:36141)
0808 02:23:00.243465 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:00.243469 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:00.243557 (+ 88us) server_negotiation.cc:407] Connection header received
0808 02:23:00.243688 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:00.243691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:00.243760 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:00.243847 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:00.244357 (+ 510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56938: BlockingRecv error: recv got EOF from 127.0.0.1:56938 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
W20260808 02:23:00.264168 25321 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:00.262896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56952 (local address 127.24.79.253:36141)
0808 02:23:00.263077 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:00.263082 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:00.263275 (+ 193us) server_negotiation.cc:407] Connection header received
0808 02:23:00.263386 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:00.263389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:00.263439 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:00.263509 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:00.264037 (+ 528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56952: BlockingRecv error: recv got EOF from 127.0.0.1:56952 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260808 02:23:00.300246 25321 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:00.298735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56964 (local address 127.24.79.253:36141)
0808 02:23:00.298938 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:00.298944 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:00.299148 (+ 204us) server_negotiation.cc:407] Connection header received
0808 02:23:00.299367 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:00.299372 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:00.299428 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:00.299502 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:00.300105 (+ 603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56964: BlockingRecv error: recv got EOF from 127.0.0.1:56964 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260808 02:23:00.368167 25321 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:00.366757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56968 (local address 127.24.79.253:36141)
0808 02:23:00.366962 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:00.366968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:00.367198 (+ 230us) server_negotiation.cc:407] Connection header received
0808 02:23:00.367361 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:00.367364 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:00.367416 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:00.367488 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:00.368028 (+ 540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56968: BlockingRecv error: recv got EOF from 127.0.0.1:56968 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260808 02:23:00.500137 25321 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:00.498619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56980 (local address 127.24.79.253:36141)
0808 02:23:00.498785 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:00.498791 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:00.498976 (+ 185us) server_negotiation.cc:407] Connection header received
0808 02:23:00.499218 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:00.499223 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:00.499291 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:00.499378 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:00.500015 (+ 637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56980: BlockingRecv error: recv got EOF from 127.0.0.1:56980 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260808 02:23:00.760010 25321 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:00.758809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56988 (local address 127.24.79.253:36141)
0808 02:23:00.759016 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:00.759021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:00.759038 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:23:00.759217 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:00.759220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:00.759271 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:00.759344 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:00.759893 (+ 549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56988: BlockingRecv error: recv got EOF from 127.0.0.1:56988 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 02:23:00.906740 25459 catalog_manager.cc:1259] Loaded cluster ID: 6aee7998ccb44e7194bed103bf1d8eae
I20260808 02:23:00.906840 25459 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7210c7a0ccaa42d5944af6fbc9d172c8: loading cluster ID for follower catalog manager: success
I20260808 02:23:00.908659 25459 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7210c7a0ccaa42d5944af6fbc9d172c8: acquiring CA information for follower catalog manager: success
I20260808 02:23:00.909929 25459 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7210c7a0ccaa42d5944af6fbc9d172c8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:23:00.935148 25480 catalog_manager.cc:1259] Loaded cluster ID: 6aee7998ccb44e7194bed103bf1d8eae
I20260808 02:23:00.935281 25480 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5823f7194dca4193beb06ed2cc7d834c: loading cluster ID for follower catalog manager: success
I20260808 02:23:00.937103 25480 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5823f7194dca4193beb06ed2cc7d834c: acquiring CA information for follower catalog manager: success
I20260808 02:23:00.938424 25480 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5823f7194dca4193beb06ed2cc7d834c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:23:01.278806 25629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:01.274562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56992 (local address 127.24.79.253:36141)
0808 02:23:01.274879 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:01.274883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:01.274901 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:23:01.274992 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:01.274995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:01.275130 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:01.275235 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:23:01.275566 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:01.276107 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:01.277007 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:01.277280 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:01.277364 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:01.277791 (+ 427us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:23:01.278300 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:23:01.278421 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:01.278528 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 02:23:01.278600 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":113,"threads_started":1}
I20260808 02:23:01.286697 25640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:01.282621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51062 (local address 127.24.79.252:37011)
0808 02:23:01.282947 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:01.282952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:01.282969 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:23:01.283066 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:01.283068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:01.283198 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:01.283320 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:23:01.283578 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:01.284107 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:01.285001 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:01.285235 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:01.285325 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:01.285665 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:23:01.286244 (+ 579us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:23:01.286378 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:01.286447 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 02:23:01.286514 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":115,"threads_started":1}
I20260808 02:23:01.294615 25629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:01.290329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57000 (local address 127.24.79.253:36141)
0808 02:23:01.290476 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:01.290482 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:01.290811 (+ 329us) server_negotiation.cc:407] Connection header received
0808 02:23:01.291027 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:01.291032 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:01.291097 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:01.291179 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:23:01.291423 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:01.292079 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:01.293070 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:01.293445 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:01.293568 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:01.294015 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:23:01.294121 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:23:01.294194 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:01.294382 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 02:23:01.294451 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 02:23:01.295164 25640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:01.290889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51078 (local address 127.24.79.252:37011)
0808 02:23:01.291033 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:01.291037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:01.291281 (+ 244us) server_negotiation.cc:407] Connection header received
0808 02:23:01.291343 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:01.291346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:01.291406 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:01.291484 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:23:01.291755 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:01.292560 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:01.293527 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:01.293756 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:01.293849 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:01.294129 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:23:01.294219 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:23:01.294279 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:01.294997 (+ 718us) server_negotiation.cc:299] Negotiation successful
0808 02:23:01.295059 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 02:23:01.295950 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:01.290389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58492 (local address 127.24.79.254:37055)
0808 02:23:01.290655 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:01.290660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:01.291093 (+ 433us) server_negotiation.cc:407] Connection header received
0808 02:23:01.291369 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:01.291373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:01.291583 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:01.291689 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:23:01.292149 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:01.292878 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:01.294060 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:01.294376 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:01.294480 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:01.294966 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:23:01.295409 (+ 443us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:23:01.295495 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:01.295717 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 02:23:01.295795 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":93,"threads_started":1}
I20260808 02:23:01.303548 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 25081
I20260808 02:23:01.320294 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 25204
I20260808 02:23:01.334612 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 25328
2026-08-08T02:23:01Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3467 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:23:01 dist-test-slave-1vmg krb5kdc[25653](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:23:01 dist-test-slave-1vmg krb5kdc[25653](info): set up 2 sockets
Aug 08 02:23:01 dist-test-slave-1vmg krb5kdc[25653](info): commencing operation
krb5kdc: starting...
W20260808 02:23:02.490698 24895 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/krb5kdc/test-user.keytab.
Aug 08 02:23:02 dist-test-slave-1vmg krb5kdc[25653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:23:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:23:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.79.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
Entry for principal kudu/127.24.79.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab.
I20260808 02:23:02.642903 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:41959
--webserver_interface=127.24.79.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:40125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:41959,127.24.79.253:37415,127.24.79.252:36055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:23:02.771346 25672 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 02:23:02.771696 25672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:23:02.771723 25672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:23:02.771764 25672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:23:02.776135 25672 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:23:02.776233 25672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:23:02.776252 25672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:23:02.776268 25672 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:23:02.776283 25672 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:23:02.782339 25672 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:41959,127.24.79.253:37415,127.24.79.252:36055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.254.keytab
--principal=kudu/127.24.79.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.254:41959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.79.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:23:02.783736 25672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:23:02.784905 25672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:23:02.794533 25681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:23:02.794533 25688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:23:02.794535 25684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:23:02.794809 25672 server_base.cc:1034] running on GCE node
I20260808 02:23:02.795686 25672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:23:02.796968 25672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:23:02.798226 25672 hybrid_clock.cc:648] HybridClock initialized: now 1786155782798197 us; error 46 us; skew 500 ppm
Aug 08 02:23:02 dist-test-slave-1vmg krb5kdc[25653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155782, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:23:02.801797 25672 init.cc:377] Logged in from keytab as kudu/127.24.79.254@KRBTEST.COM (short username kudu)
I20260808 02:23:02.802983 25672 webserver.cc:460] Webserver started at http://127.24.79.254:40535/ using document root <none> and password file <none>
I20260808 02:23:02.803632 25672 fs_manager.cc:359] Metadata directory not provided
I20260808 02:23:02.803711 25672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:23:02.803936 25672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:23:02.806627 25672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-0/data/instance:
uuid: "cec170ed328a44eaad33c7396a80baa3"
format_stamp: "Formatted at 2026-08-08 02:23:02 on dist-test-slave-1vmg"
server_key: "2a7f0d5ff54a6d7d015bcf6fbdd9497a"
server_key_iv: "d815f2768a7c709ac331d462a1ea6922"
server_key_version: "encryptionkey@0"
I20260808 02:23:02.807387 25672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-0/wal/instance:
uuid: "cec170ed328a44eaad33c7396a80baa3"
format_stamp: "Formatted at 2026-08-08 02:23:02 on dist-test-slave-1vmg"
server_key: "2a7f0d5ff54a6d7d015bcf6fbdd9497a"
server_key_iv: "d815f2768a7c709ac331d462a1ea6922"
server_key_version: "encryptionkey@0"
I20260808 02:23:02.812674 25672 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:23:02.816067 25698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:23:02.817802 25672 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:23:02.817986 25672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-0/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-0/wal
uuid: "cec170ed328a44eaad33c7396a80baa3"
format_stamp: "Formatted at 2026-08-08 02:23:02 on dist-test-slave-1vmg"
server_key: "2a7f0d5ff54a6d7d015bcf6fbdd9497a"
server_key_iv: "d815f2768a7c709ac331d462a1ea6922"
server_key_version: "encryptionkey@0"
I20260808 02:23:02.818130 25672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:23:02.853063 25672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:23:02.857157 25672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:23:02.857414 25672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:23:02.866101 25672 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.254:41959
I20260808 02:23:02.866441 25777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.254:41959 every 8 connection(s)
I20260808 02:23:02.867404 25672 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-0/data/info.pb
I20260808 02:23:02.871006 25778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:23:02.873098 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 25672
I20260808 02:23:02.873263 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-0/wal/instance
I20260808 02:23:02.873557 24895 external_mini_cluster.cc:1428] Setting key 00552775df6047572b71e54597f36350
I20260808 02:23:02.875923 25778 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41959 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.79.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.253@KRBTEST.COM" created.
Aug 08 02:23:02 dist-test-slave-1vmg krb5kdc[25653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155782, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:23:02.894259 25782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:02.878750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:41959 (local address 127.0.0.1:53836)
0808 02:23:02.879442 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:02.879509 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 02:23:02.880610 (+ 1101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:23:02.881606 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:23:02.881612 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:23:02.881991 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:23:02.882300 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:23:02.882317 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:23:02.884212 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:23:02.884216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:23:02.885281 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:23:02.885295 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:23:02.886061 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:23:02.886064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:23:02.886251 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:02.887015 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:23:02.887043 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:23:02.888844 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:23:02.890937 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:23:02.890950 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:23:02.890963 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:23:02.891234 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:23:02.891579 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:23:02.891582 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:23:02.891584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:23:02.891705 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:23:02.892377 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:23:02.892384 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:23:02.892540 (+ 156us) client_negotiation.cc:769] Sending connection context
0808 02:23:02.892704 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 02:23:02.893163 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":99,"thread_start_us":229,"threads_started":1}
I20260808 02:23:02.894259 25783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:02.879922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53836 (local address 127.24.79.254:41959)
0808 02:23:02.880325 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:02.880338 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:02.880398 (+ 60us) server_negotiation.cc:407] Connection header received
0808 02:23:02.881003 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:02.881010 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:02.881337 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:02.881578 (+ 241us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:02.882476 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:02.884019 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:02.885448 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:02.885891 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:02.886040 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:02.889019 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:23:02.889025 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:23:02.889040 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:23:02.889067 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:23:02.890781 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:02.891351 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:02.891357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:02.891359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:02.891484 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:02.891803 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:02.891805 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:02.891806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:02.892258 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:23:02.892395 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:02.892826 (+ 431us) server_negotiation.cc:299] Negotiation successful
0808 02:23:02.893069 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":117,"threads_started":1}
I20260808 02:23:02.897109 25778 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:23:02.898031 25782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:02.897696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:37415 (local address 127.0.0.1:42556)
0808 02:23:02.897864 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:02.897951 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.253:37415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260808 02:23:02.898329 25701 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.79.253:37415: connect: Connection refused (error 111)
I20260808 02:23:02.899356 25782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:02.899113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:37415 (local address 127.0.0.1:42562)
0808 02:23:02.899236 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:02.899290 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.253:37415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 02:23:02.899507 25778 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.253:37415: Network error: Client connection negotiation failed: client connection to 127.24.79.253:37415: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.253.keytab.
Entry for principal kudu/127.24.79.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.253.keytab.
I20260808 02:23:02.907198 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.253:37415
--webserver_interface=127.24.79.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.253.keytab
--principal=kudu/127.24.79.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:40125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:41959,127.24.79.253:37415,127.24.79.252:36055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:23:02.948833 25778 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } attempt: 1
I20260808 02:23:02.949862 25782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:02.949492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:37415 (local address 127.0.0.1:42570)
0808 02:23:02.949697 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:02.949770 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.253:37415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260808 02:23:02.950867 25782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:02.950543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:37415 (local address 127.0.0.1:42574)
0808 02:23:02.950727 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:02.950785 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.253:37415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260808 02:23:02.951025 25778 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.253:37415: Network error: Client connection negotiation failed: client connection to 127.24.79.253:37415: connect: Connection refused (error 111)
I20260808 02:23:03.019240 25778 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } attempt: 2
I20260808 02:23:03.020313 25782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.019944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:37415 (local address 127.0.0.1:42578)
0808 02:23:03.020150 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:03.020223 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.253:37415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260808 02:23:03.021281 25782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.020983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:37415 (local address 127.0.0.1:42586)
0808 02:23:03.021132 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:03.021193 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.253:37415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260808 02:23:03.021451 25778 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.253:37415: Network error: Client connection negotiation failed: client connection to 127.24.79.253:37415: connect: Connection refused (error 111)
W20260808 02:23:03.032440 25786 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 02:23:03.032840 25786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:23:03.032881 25786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:23:03.032943 25786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:23:03.037662 25786 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:23:03.037781 25786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:23:03.037817 25786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:23:03.037839 25786 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:23:03.037860 25786 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:23:03.044129 25786 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:41959,127.24.79.253:37415,127.24.79.252:36055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.253.keytab
--principal=kudu/127.24.79.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.253:37415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.79.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:23:03.045449 25786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:23:03.046657 25786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:23:03.056253 25792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:23:03.056253 25795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:23:03.056262 25793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:23:03.056463 25786 server_base.cc:1034] running on GCE node
I20260808 02:23:03.057333 25786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:23:03.058678 25786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:23:03.059899 25786 hybrid_clock.cc:648] HybridClock initialized: now 1786155783059856 us; error 83 us; skew 500 ppm
Aug 08 02:23:03 dist-test-slave-1vmg krb5kdc[25653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155783, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:23:03.063673 25786 init.cc:377] Logged in from keytab as kudu/127.24.79.253@KRBTEST.COM (short username kudu)
I20260808 02:23:03.064882 25786 webserver.cc:460] Webserver started at http://127.24.79.253:37903/ using document root <none> and password file <none>
I20260808 02:23:03.065573 25786 fs_manager.cc:359] Metadata directory not provided
I20260808 02:23:03.065655 25786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:23:03.065896 25786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:23:03.068399 25786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-1/data/instance:
uuid: "0910a41a0a5f46c3b1ac301f7d822f99"
format_stamp: "Formatted at 2026-08-08 02:23:03 on dist-test-slave-1vmg"
server_key: "09363174ea064608c53ad36b39c723e6"
server_key_iv: "771f608763999b2c623ce607d6a3f0e5"
server_key_version: "encryptionkey@0"
I20260808 02:23:03.069126 25786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-1/wal/instance:
uuid: "0910a41a0a5f46c3b1ac301f7d822f99"
format_stamp: "Formatted at 2026-08-08 02:23:03 on dist-test-slave-1vmg"
server_key: "09363174ea064608c53ad36b39c723e6"
server_key_iv: "771f608763999b2c623ce607d6a3f0e5"
server_key_version: "encryptionkey@0"
I20260808 02:23:03.073875 25786 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 02:23:03.077099 25808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:23:03.078811 25786 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:23:03.079022 25786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-1/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-1/wal
uuid: "0910a41a0a5f46c3b1ac301f7d822f99"
format_stamp: "Formatted at 2026-08-08 02:23:03 on dist-test-slave-1vmg"
server_key: "09363174ea064608c53ad36b39c723e6"
server_key_iv: "771f608763999b2c623ce607d6a3f0e5"
server_key_version: "encryptionkey@0"
I20260808 02:23:03.079164 25786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:23:03.111886 25786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:23:03.112648 25778 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } attempt: 3
I20260808 02:23:03.113590 25782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.113235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:37415 (local address 127.0.0.1:42596)
0808 02:23:03.113431 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:03.113501 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.253:37415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260808 02:23:03.114424 25782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.114183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:37415 (local address 127.0.0.1:42610)
0808 02:23:03.114310 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:03.114358 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.253:37415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 02:23:03.114558 25778 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.253:37415: Network error: Client connection negotiation failed: client connection to 127.24.79.253:37415: connect: Connection refused (error 111)
I20260808 02:23:03.115901 25786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:23:03.116108 25786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:23:03.124394 25786 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.253:37415
I20260808 02:23:03.124732 25878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.253:37415 every 8 connection(s)
I20260808 02:23:03.125649 25786 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-1/data/info.pb
I20260808 02:23:03.126482 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 25786
I20260808 02:23:03.126642 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-1/wal/instance
I20260808 02:23:03.127017 24895 external_mini_cluster.cc:1428] Setting key 231c1b5ec02c6c22ef10f94113ed09cc
I20260808 02:23:03.129786 25879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:23:03.134335 25879 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41959 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.79.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.79.252@KRBTEST.COM" created.
Aug 08 02:23:03 dist-test-slave-1vmg krb5kdc[25653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155783, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.253@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:23:03.152488 25783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.136852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53846 (local address 127.24.79.254:41959)
0808 02:23:03.137054 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.137058 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.138335 (+ 1277us) server_negotiation.cc:407] Connection header received
0808 02:23:03.139706 (+ 1371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.139710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.139797 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.139885 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.141834 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.142528 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.144007 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.144267 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.144372 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:03.147519 (+ 3147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:23:03.147523 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:23:03.147525 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:23:03.147560 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:23:03.149733 (+ 2173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.150434 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.150437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.150438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.150538 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.150987 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.150991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.150993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.151238 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:23:03.151339 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:03.152141 (+ 802us) server_negotiation.cc:299] Negotiation successful
0808 02:23:03.152305 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 02:23:03.153494 25883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.137308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:41959 (local address 127.0.0.1:53846)
0808 02:23:03.138084 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:03.138190 (+ 106us) client_negotiation.cc:174] Beginning negotiation
0808 02:23:03.139383 (+ 1193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:23:03.140145 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:23:03.140156 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:23:03.140732 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:23:03.141544 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:23:03.141570 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.142740 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.142743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:23:03.143833 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:23:03.143848 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.144461 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.144466 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:23:03.144666 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:03.145426 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:23:03.145448 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:23:03.147265 (+ 1817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:23:03.149913 (+ 2648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:23:03.149923 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:23:03.149943 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:23:03.150298 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:23:03.150640 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:23:03.150644 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:23:03.150646 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:23:03.150838 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:23:03.151365 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:23:03.151380 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:23:03.151758 (+ 378us) client_negotiation.cc:769] Sending connection context
0808 02:23:03.152036 (+ 278us) client_negotiation.cc:240] Negotiation successful
0808 02:23:03.152542 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":202,"threads_started":1}
I20260808 02:23:03.155624 25879 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.79.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.252.keytab.
Entry for principal kudu/127.24.79.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.252.keytab.
Aug 08 02:23:03 dist-test-slave-1vmg krb5kdc[25653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155783, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.253@KRBTEST.COM for kudu/127.24.79.253@KRBTEST.COM
I20260808 02:23:03.162853 24895 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9BOO5x/build/debug/bin/kudu
/tmp/dist-test-task9BOO5x/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.252:36055
--webserver_interface=127.24.79.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.252.keytab
--principal=kudu/127.24.79.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.79.212:40125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.79.254:41959,127.24.79.253:37415,127.24.79.252:36055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:23:03.167629 25883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.156296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:37415 (local address 127.0.0.1:42622)
0808 02:23:03.156443 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:03.156456 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:23:03.156547 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:23:03.157350 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:23:03.157353 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:23:03.157579 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:23:03.157763 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:23:03.157775 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.159024 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.159027 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:23:03.159676 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:23:03.159690 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.160516 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.160519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:23:03.160621 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:03.161192 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:23:03.161211 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:23:03.162809 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:23:03.165420 (+ 2611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:23:03.165424 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:23:03.165426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:23:03.165833 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:23:03.166257 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:23:03.166261 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:23:03.166263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:23:03.166376 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:23:03.167048 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:23:03.167051 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:23:03.167165 (+ 114us) client_negotiation.cc:769] Sending connection context
0808 02:23:03.167285 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 02:23:03.167450 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 02:23:03.168006 25885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.156410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42622 (local address 127.24.79.253:37415)
0808 02:23:03.156807 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.156818 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.156840 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:23:03.156885 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.156892 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.157077 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.157359 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.157944 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.158823 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.159838 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.160370 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.160506 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:03.163029 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:23:03.163049 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:23:03.163075 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:23:03.163110 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:23:03.165253 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.166028 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.166036 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.166037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.166148 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.166503 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.166507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.166508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.166867 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:23:03.167089 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:03.167446 (+ 357us) server_negotiation.cc:299] Negotiation successful
0808 02:23:03.167611 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":187,"threads_started":1}
I20260808 02:23:03.170526 25879 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:23:03.171516 25883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.171134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:36055 (local address 127.0.0.1:39876)
0808 02:23:03.171294 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:03.171419 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:36055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 02:23:03.171792 25811 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.79.252:36055: connect: Connection refused (error 111)
I20260808 02:23:03.172883 25883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.172583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:36055 (local address 127.0.0.1:39892)
0808 02:23:03.172734 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:03.172798 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:36055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260808 02:23:03.173161 25879 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.252:36055: Network error: Client connection negotiation failed: client connection to 127.24.79.252:36055: connect: Connection refused (error 111)
I20260808 02:23:03.222502 25879 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } attempt: 1
I20260808 02:23:03.223614 25883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.223280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:36055 (local address 127.0.0.1:39896)
0808 02:23:03.223449 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:03.223519 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:36055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 02:23:03.224534 25883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.224236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:36055 (local address 127.0.0.1:39912)
0808 02:23:03.224388 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:03.224447 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:36055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260808 02:23:03.224712 25879 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.252:36055: Network error: Client connection negotiation failed: client connection to 127.24.79.252:36055: connect: Connection refused (error 111)
I20260808 02:23:03.257800 25778 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } attempt: 4
Aug 08 02:23:03 dist-test-slave-1vmg krb5kdc[25653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155782, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for kudu/127.24.79.253@KRBTEST.COM
I20260808 02:23:03.269183 25782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.258479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:37415 (local address 127.0.0.1:42634)
0808 02:23:03.258679 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:03.258696 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:23:03.258796 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:23:03.259211 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:23:03.259214 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:23:03.259408 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:23:03.259545 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:23:03.259553 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.261346 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.261350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:23:03.261945 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:23:03.261958 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.262694 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.262699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:23:03.262888 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:03.263585 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:23:03.263599 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:23:03.265131 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:23:03.267468 (+ 2337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:23:03.267471 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:23:03.267473 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:23:03.267754 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:23:03.268116 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:23:03.268119 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:23:03.268120 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:23:03.268200 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:23:03.268675 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:23:03.268678 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:23:03.268805 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 02:23:03.268898 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 02:23:03.269039 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260808 02:23:03.269325 25885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.258591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42634 (local address 127.24.79.253:37415)
0808 02:23:03.258751 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.258756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.258778 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:23:03.258899 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.258902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.258997 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.259107 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.259704 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.261145 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.262098 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.262488 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.262617 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:03.265352 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:23:03.265356 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:23:03.265360 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:23:03.265399 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:23:03.267282 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.267885 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.267890 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.267892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.268011 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.268292 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.268295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.268297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.268563 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:23:03.268678 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:03.268987 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 02:23:03.269146 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 02:23:03.270432 25778 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:23:03.271409 25782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.271070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:36055 (local address 127.0.0.1:39914)
0808 02:23:03.271254 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:03.271320 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:36055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 02:23:03.272413 25782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.272106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:36055 (local address 127.0.0.1:39926)
0808 02:23:03.272279 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:03.272338 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:36055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 02:23:03.272578 25778 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.252:36055: Network error: Client connection negotiation failed: client connection to 127.24.79.252:36055: connect: Connection refused (error 111)
I20260808 02:23:03.292930 25879 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } attempt: 2
I20260808 02:23:03.293977 25883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.293630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:36055 (local address 127.0.0.1:39934)
0808 02:23:03.293808 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:03.293883 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:36055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 02:23:03.294967 25883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.294657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:36055 (local address 127.0.0.1:39950)
0808 02:23:03.294786 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:03.294848 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:36055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 02:23:03.295137 25879 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.252:36055: Network error: Client connection negotiation failed: client connection to 127.24.79.252:36055: connect: Connection refused (error 111)
W20260808 02:23:03.295311 25890 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 02:23:03.295661 25890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:23:03.295706 25890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:23:03.295770 25890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:23:03.300364 25890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:23:03.300462 25890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:23:03.300482 25890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:23:03.300499 25890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:23:03.300514 25890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:23:03.307189 25890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.79.212:40125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.79.254:41959,127.24.79.253:37415,127.24.79.252:36055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/krb5kdc/kudu_127.24.79.252.keytab
--principal=kudu/127.24.79.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.79.252:36055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.79.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 01:19:23 UTC on ccycloud.quasar-jszeeg.root.comops.site
build id 4573756
I20260808 02:23:03.308516 25890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:23:03.309731 25890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:23:03.319304 25904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:23:03.319317 25900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:23:03.319689 25890 server_base.cc:1034] running on GCE node
W20260808 02:23:03.319351 25898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:23:03.320482 25890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:23:03.321833 25890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:23:03.323028 25890 hybrid_clock.cc:648] HybridClock initialized: now 1786155783323000 us; error 56 us; skew 500 ppm
Aug 08 02:23:03 dist-test-slave-1vmg krb5kdc[25653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155783, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:23:03.326501 25890 init.cc:377] Logged in from keytab as kudu/127.24.79.252@KRBTEST.COM (short username kudu)
I20260808 02:23:03.327862 25890 webserver.cc:460] Webserver started at http://127.24.79.252:43989/ using document root <none> and password file <none>
I20260808 02:23:03.328642 25890 fs_manager.cc:359] Metadata directory not provided
I20260808 02:23:03.328743 25890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:23:03.329025 25890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:23:03.331743 25778 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } attempt: 1
I20260808 02:23:03.332166 25890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-2/data/instance:
uuid: "cfb3faf8f68b46a9b4133c42f292a823"
format_stamp: "Formatted at 2026-08-08 02:23:03 on dist-test-slave-1vmg"
server_key: "d02d14aa4ab21165f8197f168e54f0d0"
server_key_iv: "e860a2818091c4327abd6bb0fbb1cb14"
server_key_version: "encryptionkey@0"
I20260808 02:23:03.332655 25782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.332282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:36055 (local address 127.0.0.1:39958)
0808 02:23:03.332465 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:03.332539 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:36055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 02:23:03.332945 25890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-2/wal/instance:
uuid: "cfb3faf8f68b46a9b4133c42f292a823"
format_stamp: "Formatted at 2026-08-08 02:23:03 on dist-test-slave-1vmg"
server_key: "d02d14aa4ab21165f8197f168e54f0d0"
server_key_iv: "e860a2818091c4327abd6bb0fbb1cb14"
server_key_version: "encryptionkey@0"
I20260808 02:23:03.333422 25782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.333167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:36055 (local address 127.0.0.1:39968)
0808 02:23:03.333286 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:03.333348 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.79.252:36055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 02:23:03.333587 25778 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.79.252:36055: Network error: Client connection negotiation failed: client connection to 127.24.79.252:36055: connect: Connection refused (error 111)
I20260808 02:23:03.337713 25890 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 02:23:03.340816 25912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:23:03.342384 25890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:23:03.342552 25890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-2/data,/tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-2/wal
uuid: "cfb3faf8f68b46a9b4133c42f292a823"
format_stamp: "Formatted at 2026-08-08 02:23:03 on dist-test-slave-1vmg"
server_key: "d02d14aa4ab21165f8197f168e54f0d0"
server_key_iv: "e860a2818091c4327abd6bb0fbb1cb14"
server_key_version: "encryptionkey@0"
I20260808 02:23:03.342720 25890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:23:03.369830 25890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:23:03.373644 25890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:23:03.373837 25890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:23:03.381521 25890 rpc_server.cc:307] RPC server started. Bound to: 127.24.79.252:36055
I20260808 02:23:03.381814 25990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.79.252:36055 every 8 connection(s)
I20260808 02:23:03.382640 25890 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-2/data/info.pb
I20260808 02:23:03.383273 24895 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9BOO5x/build/debug/bin/kudu as pid 25890
I20260808 02:23:03.383417 24895 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9BOO5x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155464663577-24895-0/minicluster-data/master-2/wal/instance
I20260808 02:23:03.383692 24895 external_mini_cluster.cc:1428] Setting key fa073e8060983b4fd233553ca47edafa
I20260808 02:23:03.386312 25991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:23:03.386343 25879 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } attempt: 3
I20260808 02:23:03.391551 25991 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41959 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:23:03 dist-test-slave-1vmg krb5kdc[25653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155783, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.253@KRBTEST.COM for kudu/127.24.79.252@KRBTEST.COM
Aug 08 02:23:03 dist-test-slave-1vmg krb5kdc[25653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.252@KRBTEST.COM
I20260808 02:23:03.398849 25883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.386889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:36055 (local address 127.0.0.1:39990)
0808 02:23:03.387118 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:03.387135 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:23:03.387238 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:23:03.387730 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:23:03.387734 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:23:03.387938 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:23:03.388061 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:23:03.388068 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.389658 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.389661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:23:03.390506 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:23:03.390518 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.391534 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.391539 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:23:03.391632 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:03.392456 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:23:03.392477 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:23:03.394134 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:23:03.396686 (+ 2552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:23:03.396690 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:23:03.396692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:23:03.396975 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:23:03.397361 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:23:03.397364 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:23:03.397367 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:23:03.397465 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:23:03.398287 (+ 822us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:23:03.398291 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:23:03.398413 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 02:23:03.398506 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 02:23:03.398657 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
Aug 08 02:23:03 dist-test-slave-1vmg krb5kdc[25653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155783, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.252@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:23:03.400789 25778 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } attempt: 2
I20260808 02:23:03.401093 25994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.385207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39974 (local address 127.24.79.252:36055)
0808 02:23:03.385680 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.385701 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.385744 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:23:03.386498 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.386525 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.387038 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.387459 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.388330 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.389466 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.390677 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.391401 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.391567 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:03.395394 (+ 3827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:23:03.395398 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:23:03.395401 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:23:03.395427 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:23:03.397797 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.398990 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.398994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.398995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.399083 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.399430 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.399434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.399435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.399619 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:23:03.399800 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:03.400050 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 02:23:03.400164 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":140,"server-negotiator.queue_time_us":306,"spinlock_wait_cycles":4480,"thread_start_us":133,"threads_started":1}
I20260808 02:23:03.401093 25995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.386977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39990 (local address 127.24.79.252:36055)
0808 02:23:03.387476 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.387485 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.387500 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:23:03.387534 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.387536 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.387578 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.388175 (+ 597us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.388326 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.389501 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.390644 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.391393 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.391570 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:03.394403 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:23:03.394412 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:23:03.394433 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:23:03.394470 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:23:03.396547 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.397076 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.397081 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.397086 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.397209 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.397555 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.397559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.397561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.398137 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:23:03.399295 (+ 1158us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:03.399501 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 02:23:03.399894 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":183,"server-negotiator.queue_time_us":428,"spinlock_wait_cycles":10880,"thread_start_us":147,"threads_started":1}
I20260808 02:23:03.405592 25783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.393572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53852 (local address 127.24.79.254:41959)
0808 02:23:03.393704 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.393709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.394129 (+ 420us) server_negotiation.cc:407] Connection header received
0808 02:23:03.394354 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.394357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.394411 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.394454 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.395368 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.396139 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.397170 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.397523 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.397644 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:03.401112 (+ 3468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:23:03.401117 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:23:03.401120 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:23:03.401152 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:23:03.402703 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.403287 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.403291 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.403293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.403381 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.404481 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.404485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.404486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.404750 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:23:03.404839 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:03.405356 (+ 517us) server_negotiation.cc:299] Negotiation successful
0808 02:23:03.405471 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 02:23:03 dist-test-slave-1vmg krb5kdc[25653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155782, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.254@KRBTEST.COM for kudu/127.24.79.252@KRBTEST.COM
I20260808 02:23:03.406538 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.393514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.254:41959 (local address 127.0.0.1:53852)
0808 02:23:03.393986 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:03.394043 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 02:23:03.394233 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:23:03.394603 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:23:03.394608 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:23:03.395082 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:23:03.395229 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:23:03.395236 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.396250 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.396254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:23:03.397057 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:23:03.397069 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.397666 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.397669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:23:03.397802 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:03.398311 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:23:03.398325 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:23:03.400939 (+ 2614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:23:03.402838 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:23:03.402846 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:23:03.402851 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:23:03.403186 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:23:03.404255 (+ 1069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:23:03.404259 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:23:03.404260 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:23:03.404366 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:23:03.404912 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:23:03.404919 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:23:03.405083 (+ 164us) client_negotiation.cc:769] Sending connection context
0808 02:23:03.406099 (+ 1016us) client_negotiation.cc:240] Negotiation successful
0808 02:23:03.406336 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":1,"thread_start_us":122,"threads_started":1}
I20260808 02:23:03.406679 25879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0910a41a0a5f46c3b1ac301f7d822f99: Bootstrap starting.
I20260808 02:23:03.407871 25991 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:23:03.410137 25879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0910a41a0a5f46c3b1ac301f7d822f99: Neither blocks nor log segments found. Creating new log.
I20260808 02:23:03.410485 25782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.401237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:36055 (local address 127.0.0.1:40006)
0808 02:23:03.401382 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:03.401406 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 02:23:03.401496 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:23:03.402032 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:23:03.402035 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:23:03.402229 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:23:03.402347 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:23:03.402355 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.403442 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.403446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:23:03.404113 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:23:03.404127 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.404682 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.404687 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:23:03.404776 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:03.405259 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:23:03.405273 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:23:03.406817 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:23:03.408730 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:23:03.408733 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:23:03.408736 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:23:03.409005 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:23:03.409353 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:23:03.409357 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:23:03.409358 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:23:03.409478 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:23:03.409934 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:23:03.409938 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:23:03.410042 (+ 104us) client_negotiation.cc:769] Sending connection context
0808 02:23:03.410131 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 02:23:03.410315 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 02:23:03.411077 25994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.401693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40006 (local address 127.24.79.252:36055)
0808 02:23:03.401806 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.401810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.401821 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:23:03.401853 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.401856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.401898 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.402462 (+ 564us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.402554 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.403334 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.404247 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.404543 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.405041 (+ 498us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:03.407001 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:23:03.407004 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:23:03.407007 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:23:03.407034 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:23:03.408597 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.409137 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.409140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.409142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.409245 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.409593 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.409596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.409597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.409817 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:23:03.410674 (+ 857us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:03.410757 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 02:23:03.410879 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":38}
I20260808 02:23:03.411260 25879 log.cc:824] T 00000000000000000000000000000000 P 0910a41a0a5f46c3b1ac301f7d822f99: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:23:03 dist-test-slave-1vmg krb5kdc[25653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.254@KRBTEST.COM
I20260808 02:23:03.413846 25879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0910a41a0a5f46c3b1ac301f7d822f99: No bootstrap required, opened a new log
I20260808 02:23:03.414472 25778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3: Bootstrap starting.
Aug 08 02:23:03 dist-test-slave-1vmg krb5kdc[25653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155783, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.252@KRBTEST.COM for kudu/127.24.79.253@KRBTEST.COM
I20260808 02:23:03.417608 25778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3: Neither blocks nor log segments found. Creating new log.
I20260808 02:23:03.418124 25879 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0910a41a0a5f46c3b1ac301f7d822f99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cec170ed328a44eaad33c7396a80baa3" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41959 } } peers { permanent_uuid: "0910a41a0a5f46c3b1ac301f7d822f99" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } } peers { permanent_uuid: "cfb3faf8f68b46a9b4133c42f292a823" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } }
I20260808 02:23:03.418398 25879 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0910a41a0a5f46c3b1ac301f7d822f99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0910a41a0a5f46c3b1ac301f7d822f99, State: Initialized, Role: FOLLOWER
I20260808 02:23:03.418630 25778 log.cc:824] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3: Log is configured to *not* fsync() on all Append() calls
I20260808 02:23:03.419006 25885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.408780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42646 (local address 127.24.79.253:37415)
0808 02:23:03.408920 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.408925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.409104 (+ 179us) server_negotiation.cc:407] Connection header received
0808 02:23:03.409263 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.409266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.409315 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.409362 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.409979 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.410665 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.412364 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.412676 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.412774 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:03.415342 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:23:03.415347 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:23:03.415349 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:23:03.415382 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:23:03.416896 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.417412 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.417416 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.417418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.417505 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.417823 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.417826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.417828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.418092 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:23:03.418176 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:03.418582 (+ 406us) server_negotiation.cc:299] Negotiation successful
0808 02:23:03.418728 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 02:23:03.418789 25879 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0910a41a0a5f46c3b1ac301f7d822f99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cec170ed328a44eaad33c7396a80baa3" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41959 } } peers { permanent_uuid: "0910a41a0a5f46c3b1ac301f7d822f99" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } } peers { permanent_uuid: "cfb3faf8f68b46a9b4133c42f292a823" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } }
I20260808 02:23:03.419318 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.408393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.253:37415 (local address 127.0.0.1:42646)
0808 02:23:03.409038 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:03.409053 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:23:03.409167 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:23:03.409509 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:23:03.409512 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:23:03.409707 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:23:03.409851 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:23:03.409859 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.410965 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.410968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:23:03.412211 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:23:03.412223 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.412945 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.412949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:23:03.413063 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:03.413528 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:23:03.413543 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:23:03.415142 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:23:03.417023 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:23:03.417027 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:23:03.417029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:23:03.417296 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:23:03.417623 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:23:03.417626 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:23:03.417628 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:23:03.417720 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:23:03.418257 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:23:03.418260 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:23:03.418382 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 02:23:03.418820 (+ 438us) client_negotiation.cc:240] Negotiation successful
0808 02:23:03.419031 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":109,"threads_started":1}
I20260808 02:23:03.419800 25783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.404564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53858 (local address 127.24.79.254:41959)
0808 02:23:03.405766 (+ 1202us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.405770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.405782 (+ 12us) server_negotiation.cc:407] Connection header received
0808 02:23:03.405841 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.405843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.405885 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.405959 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.407140 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.408316 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.409506 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.409817 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.409898 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:03.414131 (+ 4233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:23:03.414136 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:23:03.414139 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:23:03.414172 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:23:03.416640 (+ 2468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.417246 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.417250 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.417252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.417349 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.417766 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.417770 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.417772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.418022 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:23:03.418191 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:03.418837 (+ 646us) server_negotiation.cc:299] Negotiation successful
0808 02:23:03.418998 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1130,"thread_start_us":99,"threads_started":1}
I20260808 02:23:03.420135 25991 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:23:03.420761 25879 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0910a41a0a5f46c3b1ac301f7d822f99 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:23:03.421166 25778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3: No bootstrap required, opened a new log
Aug 08 02:23:03 dist-test-slave-1vmg krb5kdc[25653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155783, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.79.252@KRBTEST.COM for kudu/127.24.79.252@KRBTEST.COM
I20260808 02:23:03.424662 25786 master_runner.cc:186] Error getting master registration for 127.24.79.254:41959: OK, instance_id { permanent_uuid: "cec170ed328a44eaad33c7396a80baa3" instance_seqno: 1786155782863558 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 02:23:03.428072 25786 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 08 02:23:03 dist-test-slave-1vmg krb5kdc[25653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.79.253@KRBTEST.COM
I20260808 02:23:03.430436 25994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.421140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40022 (local address 127.24.79.252:36055)
0808 02:23:03.421235 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.421239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.421250 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:23:03.421284 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.421287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.421329 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.421392 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.421974 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.422680 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.423773 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.424085 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.424165 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:03.426903 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:23:03.426907 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:23:03.426910 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:23:03.426985 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:23:03.428603 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.429076 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.429079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.429081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.429174 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.429461 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.429465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.429466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.429698 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:23:03.429773 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:03.430126 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 02:23:03.430269 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 02:23:03.430954 25999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.420671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.79.252:36055 (local address 127.0.0.1:40022)
0808 02:23:03.420906 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 02:23:03.420920 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:23:03.421016 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:23:03.421508 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:23:03.421511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:23:03.421717 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:23:03.421868 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:23:03.421875 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.422826 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.422829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:23:03.423671 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:23:03.423682 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.424303 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.424306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:23:03.424395 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:03.424887 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:23:03.424902 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:23:03.426756 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:23:03.428736 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:23:03.428739 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:23:03.428741 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:23:03.428979 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:23:03.429287 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:23:03.429290 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:23:03.429292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:23:03.429375 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:23:03.429828 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:23:03.429831 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:23:03.429937 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 02:23:03.430592 (+ 655us) client_negotiation.cc:240] Negotiation successful
0808 02:23:03.430747 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158}
I20260808 02:23:03.430716 25778 raft_consensus.cc:348] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cec170ed328a44eaad33c7396a80baa3" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41959 } } peers { permanent_uuid: "0910a41a0a5f46c3b1ac301f7d822f99" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } } peers { permanent_uuid: "cfb3faf8f68b46a9b4133c42f292a823" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } }
I20260808 02:23:03.431036 25778 raft_consensus.cc:729] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cec170ed328a44eaad33c7396a80baa3, State: Initialized, Role: FOLLOWER
I20260808 02:23:03.431794 25778 consensus_queue.cc:260] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cec170ed328a44eaad33c7396a80baa3" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41959 } } peers { permanent_uuid: "0910a41a0a5f46c3b1ac301f7d822f99" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } } peers { permanent_uuid: "cfb3faf8f68b46a9b4133c42f292a823" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } }
I20260808 02:23:03.433503 25778 sys_catalog.cc:564] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:23:03.434875 25991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfb3faf8f68b46a9b4133c42f292a823: Bootstrap starting.
W20260808 02:23:03.435425 26021 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0910a41a0a5f46c3b1ac301f7d822f99: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:23:03.435504 26021 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:23:03.435755 26003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0910a41a0a5f46c3b1ac301f7d822f99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cec170ed328a44eaad33c7396a80baa3" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41959 } } peers { permanent_uuid: "0910a41a0a5f46c3b1ac301f7d822f99" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } } peers { permanent_uuid: "cfb3faf8f68b46a9b4133c42f292a823" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } } }
I20260808 02:23:03.435910 26003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0910a41a0a5f46c3b1ac301f7d822f99 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:23:03.436865 25885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.422051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42648 (local address 127.24.79.253:37415)
0808 02:23:03.422191 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.422196 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.422439 (+ 243us) server_negotiation.cc:407] Connection header received
0808 02:23:03.422611 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.422614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.422671 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.422723 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.423729 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.424413 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.425954 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.426300 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.426381 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:03.430362 (+ 3981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:23:03.430367 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:23:03.430370 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:23:03.430402 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:23:03.433031 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.433663 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.433667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.433669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.433785 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.434129 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.434133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.434134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.435728 (+ 1594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:23:03.435937 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:03.436540 (+ 603us) server_negotiation.cc:299] Negotiation successful
0808 02:23:03.436693 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 02:23:03.438127 25991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfb3faf8f68b46a9b4133c42f292a823: Neither blocks nor log segments found. Creating new log.
I20260808 02:23:03.439301 25991 log.cc:824] T 00000000000000000000000000000000 P cfb3faf8f68b46a9b4133c42f292a823: Log is configured to *not* fsync() on all Append() calls
W20260808 02:23:03.440187 26045 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:23:03.440294 26045 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:23:03.440608 26025 sys_catalog.cc:455] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cec170ed328a44eaad33c7396a80baa3" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41959 } } peers { permanent_uuid: "0910a41a0a5f46c3b1ac301f7d822f99" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } } peers { permanent_uuid: "cfb3faf8f68b46a9b4133c42f292a823" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } } }
I20260808 02:23:03.440752 26025 sys_catalog.cc:458] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:23:03.441725 24895 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:23:03.443181 25741 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "cec170ed328a44eaad33c7396a80baa3"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53858
I20260808 02:23:03.443476 25741 raft_consensus.cc:482] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 02:23:03.443598 25741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:23:03.445076 25741 raft_consensus.cc:504] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cec170ed328a44eaad33c7396a80baa3" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41959 } } peers { permanent_uuid: "0910a41a0a5f46c3b1ac301f7d822f99" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } } peers { permanent_uuid: "cfb3faf8f68b46a9b4133c42f292a823" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } }
I20260808 02:23:03.446210 25741 leader_election.cc:288] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3 [CANDIDATE]: Term 1 election: Requested vote from peers 0910a41a0a5f46c3b1ac301f7d822f99 (127.24.79.253:37415), cfb3faf8f68b46a9b4133c42f292a823 (127.24.79.252:36055)
I20260808 02:23:03.442584 25672 master_runner.cc:186] Error getting master registration for 127.24.79.252:36055: OK, instance_id { permanent_uuid: "cfb3faf8f68b46a9b4133c42f292a823" instance_seqno: 1786155783379427 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 02:23:03.446839 25672 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:23:03.447754 25842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cec170ed328a44eaad33c7396a80baa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0910a41a0a5f46c3b1ac301f7d822f99"
I20260808 02:23:03.448050 25842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0910a41a0a5f46c3b1ac301f7d822f99 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:23:03.449517 25842 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0910a41a0a5f46c3b1ac301f7d822f99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cec170ed328a44eaad33c7396a80baa3 in term 1.
I20260808 02:23:03.450012 25701 leader_election.cc:302] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0910a41a0a5f46c3b1ac301f7d822f99, cec170ed328a44eaad33c7396a80baa3; no voters:
I20260808 02:23:03.450395 26025 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:23:03.450640 26025 raft_consensus.cc:686] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3 [term 1 LEADER]: Becoming Leader. State: Replica: cec170ed328a44eaad33c7396a80baa3, State: Running, Role: LEADER
I20260808 02:23:03.451120 26025 consensus_queue.cc:237] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cec170ed328a44eaad33c7396a80baa3" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41959 } } peers { permanent_uuid: "0910a41a0a5f46c3b1ac301f7d822f99" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } } peers { permanent_uuid: "cfb3faf8f68b46a9b4133c42f292a823" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } }
I20260808 02:23:03.451201 25991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfb3faf8f68b46a9b4133c42f292a823: No bootstrap required, opened a new log
I20260808 02:23:03.453408 26025 sys_catalog.cc:455] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cec170ed328a44eaad33c7396a80baa3. Latest consensus state: current_term: 1 leader_uuid: "cec170ed328a44eaad33c7396a80baa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cec170ed328a44eaad33c7396a80baa3" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41959 } } peers { permanent_uuid: "0910a41a0a5f46c3b1ac301f7d822f99" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } } peers { permanent_uuid: "cfb3faf8f68b46a9b4133c42f292a823" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } } }
I20260808 02:23:03.453567 26025 sys_catalog.cc:458] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3 [sys.catalog]: This master's current role is: LEADER
I20260808 02:23:03.459292 26065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:23:03.459018 25991 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfb3faf8f68b46a9b4133c42f292a823 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cec170ed328a44eaad33c7396a80baa3" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41959 } } peers { permanent_uuid: "0910a41a0a5f46c3b1ac301f7d822f99" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } } peers { permanent_uuid: "cfb3faf8f68b46a9b4133c42f292a823" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } }
I20260808 02:23:03.459524 25991 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfb3faf8f68b46a9b4133c42f292a823 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfb3faf8f68b46a9b4133c42f292a823, State: Initialized, Role: FOLLOWER
I20260808 02:23:03.460135 25991 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfb3faf8f68b46a9b4133c42f292a823 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cec170ed328a44eaad33c7396a80baa3" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41959 } } peers { permanent_uuid: "0910a41a0a5f46c3b1ac301f7d822f99" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } } peers { permanent_uuid: "cfb3faf8f68b46a9b4133c42f292a823" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } }
I20260808 02:23:03.460511 26065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:23:03.462632 25953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cec170ed328a44eaad33c7396a80baa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cfb3faf8f68b46a9b4133c42f292a823"
I20260808 02:23:03.462980 25953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfb3faf8f68b46a9b4133c42f292a823 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:23:03.464519 25953 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cfb3faf8f68b46a9b4133c42f292a823 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cec170ed328a44eaad33c7396a80baa3 in term 1.
I20260808 02:23:03.462575 25991 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfb3faf8f68b46a9b4133c42f292a823 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:23:03.475498 26066 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb3faf8f68b46a9b4133c42f292a823 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cec170ed328a44eaad33c7396a80baa3" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41959 } } peers { permanent_uuid: "0910a41a0a5f46c3b1ac301f7d822f99" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } } peers { permanent_uuid: "cfb3faf8f68b46a9b4133c42f292a823" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } } }
I20260808 02:23:03.475675 26066 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb3faf8f68b46a9b4133c42f292a823 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 02:23:03.479889 26083 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cfb3faf8f68b46a9b4133c42f292a823: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 02:23:03.479943 25783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.450379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53868 (local address 127.24.79.254:41959)
0808 02:23:03.450501 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.450506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.453889 (+ 3383us) server_negotiation.cc:407] Connection header received
0808 02:23:03.455094 (+ 1205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.455098 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.455157 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.455212 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.459535 (+ 4323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.460235 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.465264 (+ 5029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.465601 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.465684 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:03.471202 (+ 5518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:23:03.471206 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:23:03.471209 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:23:03.471244 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:23:03.473698 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.475181 (+ 1483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.475185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.475187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.475302 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.477707 (+ 2405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.477712 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.477714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.477974 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:23:03.478095 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:03.479620 (+ 1525us) server_negotiation.cc:299] Negotiation successful
0808 02:23:03.479784 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 02:23:03.479979 26083 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:23:03.480774 25994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.451064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40028 (local address 127.24.79.252:36055)
0808 02:23:03.451181 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.451185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.452791 (+ 1606us) server_negotiation.cc:407] Connection header received
0808 02:23:03.452992 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.452996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.453051 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.453105 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.457481 (+ 4376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.458320 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.464094 (+ 5774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.464426 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.464519 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:03.470684 (+ 6165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:23:03.470689 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:23:03.470692 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:23:03.470729 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:23:03.472865 (+ 2136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.477134 (+ 4269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.477138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.477140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.477248 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.478432 (+ 1184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.478436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.478438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.478666 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:23:03.478774 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:03.480504 (+ 1730us) server_negotiation.cc:299] Negotiation successful
0808 02:23:03.480636 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 02:23:03.482656 25953 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cfb3faf8f68b46a9b4133c42f292a823 [term 1 FOLLOWER]: Refusing update from remote peer cec170ed328a44eaad33c7396a80baa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:23:03.483079 25842 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0910a41a0a5f46c3b1ac301f7d822f99 [term 1 FOLLOWER]: Refusing update from remote peer cec170ed328a44eaad33c7396a80baa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:23:03.483366 26025 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfb3faf8f68b46a9b4133c42f292a823" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:23:03.483798 26025 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0910a41a0a5f46c3b1ac301f7d822f99" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:23:03.484644 25885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:03.452151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42650 (local address 127.24.79.253:37415)
0808 02:23:03.452268 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.452273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.454114 (+ 1841us) server_negotiation.cc:407] Connection header received
0808 02:23:03.456329 (+ 2215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.456333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.456395 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.456450 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.460730 (+ 4280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.461581 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.466323 (+ 4742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:03.466679 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:03.466771 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:03.474733 (+ 7962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:23:03.474736 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:23:03.474739 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:23:03.474771 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:23:03.476568 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.479116 (+ 2548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.479120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.479122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.479236 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:23:03.482462 (+ 3226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:23:03.482466 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:23:03.482468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:23:03.482749 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:23:03.482866 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:03.484351 (+ 1485us) server_negotiation.cc:299] Negotiation successful
0808 02:23:03.484492 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 02:23:03.487879 26066 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb3faf8f68b46a9b4133c42f292a823 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cec170ed328a44eaad33c7396a80baa3. Latest consensus state: current_term: 1 leader_uuid: "cec170ed328a44eaad33c7396a80baa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cec170ed328a44eaad33c7396a80baa3" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41959 } } peers { permanent_uuid: "0910a41a0a5f46c3b1ac301f7d822f99" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } } peers { permanent_uuid: "cfb3faf8f68b46a9b4133c42f292a823" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } } }
I20260808 02:23:03.488023 26066 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb3faf8f68b46a9b4133c42f292a823 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:23:03.488471 26003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0910a41a0a5f46c3b1ac301f7d822f99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cec170ed328a44eaad33c7396a80baa3. Latest consensus state: current_term: 1 leader_uuid: "cec170ed328a44eaad33c7396a80baa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cec170ed328a44eaad33c7396a80baa3" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41959 } } peers { permanent_uuid: "0910a41a0a5f46c3b1ac301f7d822f99" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } } peers { permanent_uuid: "cfb3faf8f68b46a9b4133c42f292a823" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } } }
I20260808 02:23:03.488616 26003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0910a41a0a5f46c3b1ac301f7d822f99 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:23:03.497941 26060 sys_catalog.cc:455] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cec170ed328a44eaad33c7396a80baa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cec170ed328a44eaad33c7396a80baa3" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41959 } } peers { permanent_uuid: "0910a41a0a5f46c3b1ac301f7d822f99" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } } peers { permanent_uuid: "cfb3faf8f68b46a9b4133c42f292a823" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } } }
I20260808 02:23:03.498131 26060 sys_catalog.cc:458] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3 [sys.catalog]: This master's current role is: LEADER
I20260808 02:23:03.499485 26003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0910a41a0a5f46c3b1ac301f7d822f99 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cec170ed328a44eaad33c7396a80baa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cec170ed328a44eaad33c7396a80baa3" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41959 } } peers { permanent_uuid: "0910a41a0a5f46c3b1ac301f7d822f99" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } } peers { permanent_uuid: "cfb3faf8f68b46a9b4133c42f292a823" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } } }
I20260808 02:23:03.499639 26003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0910a41a0a5f46c3b1ac301f7d822f99 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:23:03.500371 26089 mvcc.cc:205] Tried to move back new op lower bound from 7316094089137786880 to 7316094089020211200. Current Snapshot: MvccSnapshot[applied={T|T < 7316094089137786880}]
I20260808 02:23:03.506471 26065 catalog_manager.cc:1347] Generated new cluster ID: 51db041be7ba43f18a9719a5ab47f744
I20260808 02:23:03.506577 26065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:23:03.515751 26025 sys_catalog.cc:455] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cec170ed328a44eaad33c7396a80baa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cec170ed328a44eaad33c7396a80baa3" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41959 } } peers { permanent_uuid: "0910a41a0a5f46c3b1ac301f7d822f99" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } } peers { permanent_uuid: "cfb3faf8f68b46a9b4133c42f292a823" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } } }
I20260808 02:23:03.515941 26025 sys_catalog.cc:458] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3 [sys.catalog]: This master's current role is: LEADER
I20260808 02:23:03.518081 26088 mvcc.cc:205] Tried to move back new op lower bound from 7316094089137786880 to 7316094089020211200. Current Snapshot: MvccSnapshot[applied={T|T < 7316094089137786880}]
I20260808 02:23:03.518517 26066 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb3faf8f68b46a9b4133c42f292a823 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cec170ed328a44eaad33c7396a80baa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cec170ed328a44eaad33c7396a80baa3" member_type: VOTER last_known_addr { host: "127.24.79.254" port: 41959 } } peers { permanent_uuid: "0910a41a0a5f46c3b1ac301f7d822f99" member_type: VOTER last_known_addr { host: "127.24.79.253" port: 37415 } } peers { permanent_uuid: "cfb3faf8f68b46a9b4133c42f292a823" member_type: VOTER last_known_addr { host: "127.24.79.252" port: 36055 } } }
I20260808 02:23:03.518666 26066 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb3faf8f68b46a9b4133c42f292a823 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:23:03.536867 26065 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:23:03.538339 26065 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:23:03.584841 26065 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cec170ed328a44eaad33c7396a80baa3: Generated new TSK 0
I20260808 02:23:03.585945 26065 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 02:23:03.628553 25994 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:03.626499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40038 (local address 127.24.79.252:36055)
0808 02:23:03.626623 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.626627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.627245 (+ 618us) server_negotiation.cc:407] Connection header received
0808 02:23:03.627476 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.627480 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.627535 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.627593 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.628410 (+ 817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40038: BlockingRecv error: recv got EOF from 127.0.0.1:40038 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 02:23:03.630523 25783 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:03.626156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53882 (local address 127.24.79.254:41959)
0808 02:23:03.626395 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.626401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.629197 (+ 2796us) server_negotiation.cc:407] Connection header received
0808 02:23:03.629426 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.629429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.629493 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.629553 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.630366 (+ 813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53882: BlockingRecv error: recv got EOF from 127.0.0.1:53882 (error 108)
Metrics: {"server-negotiator.queue_time_us":143}
W20260808 02:23:03.632534 25885 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:03.626809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42660 (local address 127.24.79.253:37415)
0808 02:23:03.626977 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.626982 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.631245 (+ 4263us) server_negotiation.cc:407] Connection header received
0808 02:23:03.631482 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.631486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.631564 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.631629 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.632378 (+ 749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42660: BlockingRecv error: recv got EOF from 127.0.0.1:42660 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 02:23:03.638106 25885 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:03.636337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42672 (local address 127.24.79.253:37415)
0808 02:23:03.636469 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.636474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.636778 (+ 304us) server_negotiation.cc:407] Connection header received
0808 02:23:03.637075 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.637080 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.637181 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.637251 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.637966 (+ 715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42672: BlockingRecv error: recv got EOF from 127.0.0.1:42672 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 02:23:03.643427 25885 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:03.642243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42674 (local address 127.24.79.253:37415)
0808 02:23:03.642377 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.642381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.642396 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:23:03.642460 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.642463 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.642518 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.642604 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.643289 (+ 685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42674: BlockingRecv error: recv got EOF from 127.0.0.1:42674 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 02:23:03.650692 25885 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:03.648608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42688 (local address 127.24.79.253:37415)
0808 02:23:03.648750 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.648755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.649469 (+ 714us) server_negotiation.cc:407] Connection header received
0808 02:23:03.649709 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.649714 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.649782 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.649842 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.650547 (+ 705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42688: BlockingRecv error: recv got EOF from 127.0.0.1:42688 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 02:23:03.659660 25885 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:03.658167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42698 (local address 127.24.79.253:37415)
0808 02:23:03.658383 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.658388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.658620 (+ 232us) server_negotiation.cc:407] Connection header received
0808 02:23:03.658759 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.658762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.658828 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.658984 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.659518 (+ 534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42698: BlockingRecv error: recv got EOF from 127.0.0.1:42698 (error 108)
Metrics: {"server-negotiator.queue_time_us":126}
W20260808 02:23:03.671830 25885 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:03.670447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42710 (local address 127.24.79.253:37415)
0808 02:23:03.670591 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.670596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.670806 (+ 210us) server_negotiation.cc:407] Connection header received
0808 02:23:03.670974 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.670978 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.671044 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.671152 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.671682 (+ 530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42710: BlockingRecv error: recv got EOF from 127.0.0.1:42710 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260808 02:23:03.692014 25885 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:03.690674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42718 (local address 127.24.79.253:37415)
0808 02:23:03.690826 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.690831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.691002 (+ 171us) server_negotiation.cc:407] Connection header received
0808 02:23:03.691149 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.691152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.691215 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.691321 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.691881 (+ 560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42718: BlockingRecv error: recv got EOF from 127.0.0.1:42718 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260808 02:23:03.728127 25885 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:03.726813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42722 (local address 127.24.79.253:37415)
0808 02:23:03.727047 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.727052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.727174 (+ 122us) server_negotiation.cc:407] Connection header received
0808 02:23:03.727345 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.727348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.727399 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.727475 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.727998 (+ 523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42722: BlockingRecv error: recv got EOF from 127.0.0.1:42722 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
W20260808 02:23:03.795941 25885 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:03.794632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42726 (local address 127.24.79.253:37415)
0808 02:23:03.794780 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.794785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.794972 (+ 187us) server_negotiation.cc:407] Connection header received
0808 02:23:03.795158 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.795162 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.795216 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.795306 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.795809 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42726: BlockingRecv error: recv got EOF from 127.0.0.1:42726 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260808 02:23:03.927659 25885 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:03.926221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42742 (local address 127.24.79.253:37415)
0808 02:23:03.926408 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:03.926412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:03.926561 (+ 149us) server_negotiation.cc:407] Connection header received
0808 02:23:03.926691 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:03.926694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:03.926752 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:03.926851 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:03.927546 (+ 695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42742: BlockingRecv error: recv got EOF from 127.0.0.1:42742 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 02:23:04.187249 25885 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:23:04.186056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42746 (local address 127.24.79.253:37415)
0808 02:23:04.186235 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:04.186239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:04.186279 (+ 40us) server_negotiation.cc:407] Connection header received
0808 02:23:04.186461 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:04.186464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:04.186519 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:04.186593 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:23:04.187131 (+ 538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42746: BlockingRecv error: recv got EOF from 127.0.0.1:42746 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 02:23:04.437230 26021 catalog_manager.cc:1259] Loaded cluster ID: 51db041be7ba43f18a9719a5ab47f744
I20260808 02:23:04.437450 26021 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0910a41a0a5f46c3b1ac301f7d822f99: loading cluster ID for follower catalog manager: success
I20260808 02:23:04.439507 26021 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0910a41a0a5f46c3b1ac301f7d822f99: acquiring CA information for follower catalog manager: success
I20260808 02:23:04.441256 26021 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0910a41a0a5f46c3b1ac301f7d822f99: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:23:04.481510 26083 catalog_manager.cc:1259] Loaded cluster ID: 51db041be7ba43f18a9719a5ab47f744
I20260808 02:23:04.481613 26083 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cfb3faf8f68b46a9b4133c42f292a823: loading cluster ID for follower catalog manager: success
I20260808 02:23:04.483597 26083 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cfb3faf8f68b46a9b4133c42f292a823: acquiring CA information for follower catalog manager: success
I20260808 02:23:04.484995 26083 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cfb3faf8f68b46a9b4133c42f292a823: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:23:04.706298 26200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:04.701693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50820 (local address 127.24.79.253:37415)
0808 02:23:04.701947 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:04.701951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:04.701992 (+ 41us) server_negotiation.cc:407] Connection header received
0808 02:23:04.702143 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:04.702145 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:04.702295 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:04.702400 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:23:04.702790 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:04.703389 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:04.704433 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:04.704723 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:04.704823 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:04.705258 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:23:04.705820 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:23:04.705975 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:04.706038 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 02:23:04.706118 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":82,"threads_started":1}
I20260808 02:23:04.713828 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:04.709722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46392 (local address 127.24.79.252:36055)
0808 02:23:04.709998 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:04.710003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:04.710022 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:23:04.710279 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:04.710282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:04.710416 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:04.710523 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:23:04.710794 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:04.711375 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:04.712244 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:04.712473 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:04.712563 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:04.712900 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:23:04.713397 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:23:04.713518 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:04.713597 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 02:23:04.713662 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":79,"threads_started":1}
I20260808 02:23:04.721428 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:04.717441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46408 (local address 127.24.79.252:36055)
0808 02:23:04.717569 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:04.717573 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:04.717646 (+ 73us) server_negotiation.cc:407] Connection header received
0808 02:23:04.717904 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:04.717909 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:04.717972 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:04.718052 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:23:04.718391 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:04.719136 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:04.720297 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:04.720528 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:04.720621 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:04.720962 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:23:04.721046 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:23:04.721108 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:04.721220 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 02:23:04.721282 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 02:23:04.722484 26200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:04.718001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50826 (local address 127.24.79.253:37415)
0808 02:23:04.718145 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:04.718149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:04.718372 (+ 223us) server_negotiation.cc:407] Connection header received
0808 02:23:04.718559 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:04.718562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:04.718625 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:04.718711 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:23:04.718996 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:04.719785 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:04.720960 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:04.721295 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:04.721396 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:04.721900 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:23:04.722020 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:23:04.722069 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:04.722270 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 02:23:04.722347 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:23:04.722856 26217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:23:04.717329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55030 (local address 127.24.79.254:41959)
0808 02:23:04.717700 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0808 02:23:04.717704 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:23:04.717935 (+ 231us) server_negotiation.cc:407] Connection header received
0808 02:23:04.718183 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:23:04.718186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:23:04.718406 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:23:04.718510 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:23:04.718866 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:04.719529 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:04.720796 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:23:04.721135 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:23:04.721234 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:23:04.721929 (+ 695us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:23:04.722261 (+ 332us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:23:04.722364 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 02:23:04.722631 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 02:23:04.722712 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":79,"threads_started":1}
I20260808 02:23:04.726243 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 25672
I20260808 02:23:04.742663 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 25786
I20260808 02:23:04.758143 24895 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9BOO5x/build/debug/bin/kudu with pid 25890
2026-08-08T02:23:04Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3428 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6896 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (320095 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST